Ming-Li Wang wrote:Sorry for not making it clear in the first place: by "it didn't work" I meant nothing happened after I double-clicked the archive in the "Open Archive..." dialog (or clicked the "Open" button after selecting the archive). The dialog would close and that's it. Nothing else happened.

Tried it a few times just now, and made sure to wait more than 15 min. for it. Still nothing. Had to double-click in the Finder to open in able to conduct the next test.

Opening a document in QRecall (heck, practically every Cocoa app) via the open command vs. double-clicking a document in the Finder uses almost identical code; there are only tiny differences. So for one to work and the other not is profoundly strange.

If a stale lock is the cause, wouldn't it also prevent the archive from opening right away from Finder?

No, this has nothing to do with stale locks. (If it was a stale lock, you'd see a dialog that the archive was being modified.) What I do suspect is happening is that the archive browser window is failing to display for some reason, but the archive is still being opened. Then, as long as the QRecall app is running, this prevents other actions from modifying the archive—very much an active lock.

That gave me an idea: after canceling the delete job and quitting QR to close everything, I reopened QR and made a manual capture from the Actions window. The capture waited about 5 min. before going into action. The is probably due to the stale lock you mentioned.

That sort of confirms my suspicions that the failed attempt to open the archive earlier was keeping the archive locked for modifications. By quoting the QRecall app, the lock on the archive was abandoned, allowing the other actions to (eventually) break the lock and proceed.

I recommend running a verify to determine if the archive is OK or needs repair. I can't tell from the information here.

Also, send a diagnostic report form the MBP so I can look into the details.

Ming-Li Wang wrote:Today when trying to open an archive from "File - Open Archive ..." dialog, it didn't work (it's been a while since I last opened the archive, so it's no longer on the recent archives list). Double clicking on the archive in Finder did work, but then it can't be closed, either by clicking on the window control button or with the "File - Close" command (or cmd-W, for that matter).

What exactly do you mean by "it didn't work"? Looking at the logs, I don't see anything that would indicate a problem opening the archive. Was the window just blank? Was it taking forever to display the layers? ... ?

Had to quit QR to close it. I've tried it several times with the same result.

Looking at the log, I did discover a bug that can occur when closing an archive (either to run a command or to close the window). That bug, inadvertently, left the shared read mode set for the archive, which is I think the cause of your later difficulties.

I even tried it once on another desktop (running the same beta version, but on Yosemite); same result.

Yes, once the archive has a stale lock on it, any QRecall process that tries to access it is going to get stuck.

The problem is not limited to opening and closing. I can't delete any item on it, nor can I remove any layer, merge layers, compact the archive or combine volumes. (The archive has two volumes of the same source directory after I installed El Capitan, so I want to combine them.) The action would appear to be starting, but would be "waiting for archive" to no end.

Be patient. Once an archive acquires an orphaned access lock, it take a while for QRecall to determine that the lock is stale and reset it. It does this automatically, but it takes about 10 minutes for QRecall assure itself that there are no other processes accessing the archive at a the same time.

After clicking "Stop" to cancel the action, there's an entry in the log that says "Problem encountered while closing archive", so it appeared to be related to the closing problem after all.

That's actually a message from the browser, and is related to the bug I mentioned earlier (which I just fixed).

Verifications (numerous times) found no error. Regular captures seem to be fine as well. I also successfully compacted it from the Actions window without opening the archive, but the action waited more than 10 min. before really doing anything.

I suspect all is well now. You waited the magic 10 minutes for QRecall to determine that it was safe to access the archive. Once it had, the access lock for the archive was reset and actions will now proceed normally.

I have never benchmarked FileVault, so i can't comment on its performance relative to QRecall's. I would expect, however, that they would be comparable. QRecall uses the encryption engine built into OS X, which is what I assume FileVault uses.

There are several aspects of QRecall's encryption that differ from whole disk encryption:

QRecall allows archives to be encrypted on volumes that don't support encryption.

Each archive has its own key. A single volume can contain multiple archives, each encrypted with a different key.

Other processes that have read access to the archive won't see any plaintext data.

An archive key is a completely random AES key, not derived from a password, so it has maximum entropy. This means that the encryption is stronger than the password (and immune to password crackers).

Because the key isn't based on a password, the archive's password can be changed without re-encrypting the archive.

Adrian Chapman wrote:It is always a Macbook archive that shows up as not being run for x days, but when I check the actions have been running correctly. I should add that all my verify, merge and compact actions run from my iMac.

Adrian,

Can you clarify a few things? Are you capturing to the same archive on the NAS, or separate archives for the iMac and Macbook? Is the status message you're talking about the one on the Macbook or the iMac? Does the iMac and Macbook show different status message for the same archive? Is the NAS connected to the Macbook all of the time, or only during the capture? Does the Macbook capture occur when it's logged in or out?

When archives are shared across networks, keeping the status information up-to-date across all participants gets a little tricky.

David Ramsey wrote:There was an interrupted backup 10 days ago. But every backup since then has gone perfectly (I checked the log file).

I think I've discovered where the confusion lies. You have (or had) two archives. You were capturing to the older archive, but then you deleted all of the actions for that archive, created a new archive (with a different name), and created new actions for the new archive. The new archive is working just fine, but the old archive is no longer being captured to.

That's the reason it's status is giving you that warning.

Now, if the old archive is really dead and/or gone, then you can remove it from the status window by right/control+clicking on its status and choosing the Forget command. The status window will stop tracking that archive (unless you perform another action on it, then it will start monitoring it again).

Very odd.

If you haven't already done so, please send a diagnostic report. Then, if you have the time, please locate the following files and send copies to support@qrecall.com:

1) The files in the folder ~/Library/Preferences/QRecall/Status

2) Locate the archive that's being captured, right/control+click on its icon and choose Show Package Contents. Find its status.plist file.

Send me those files and I'll have a look.

Bob Tyson wrote:James, here a new report. I read your latest reply in the Forum. The Status Window now shows red for the 1TB-xx archive, on the first line with the note ?Captured 2 days ago?.

That indicates that this archive hasn't performed a capture in 2 days. QRecall monitors the frequency at which you normally capture items to an archive, and raises a warning (in the status window) if that frequency suddenly drops off. In this case, I suspect that you regularly capture to this archive more than once a day. The status window is saying that nothing has been captured in over 2 days now, which might indicate a problem.

For a damaged layer, if I perform a merge, does this mean that only latest versions of files will be retained?

That's correct. Each layer embodies the changes that occurred since the previous layer was captured. When you merge two or more layers, only the latest version of those items is retained in the resulting layer.

Our concern is that files from earlier layers be retained (so long as not damaged) in order to maintain the integrity of the archive and not lose older 'original' photographic images.

Since you didn't use any of the special repair options, none of the items in the layer are damaged. Said another way, following a repair (with default options) all the items remaining in a layer will be intact and complete. The issue is items that might have been lost due to corrupted data.

In the browser, you can narrow the focus to a single layer using the layer shades and review just the items in that layer. (Tip: select the layer and choose "Shade Unselected Layers".) The items you now see in the browser are just the items captured in that layer. They are all fine, but there may be items that are missing because they were lost during the repair—and we have no idea what those are, because that data was corrupted.

The rest of your log looks good. I see nothing alarming.

One last remark, I truly appreciate your patience and support. Perhaps it helps that I'm posting at hours when you may be sleeping or at least not at your screen, and vice versa. Kind of moderates the pace?

It does, but maybe that's a good thing.

Bob Tyson wrote:Monitor 2015-11-28 03:15:41 Minutia Capture history for archive 1TB_FOTO_ARCHIVIO_PORTABLE_HD.alias appears to have been deleted
Monitor 2015-11-28 03:21:37 Minutia Capture history for archive 1TB_FOTO_ARCHIVIO_PORTABLE_HD.alias appears to have been deleted
Monitor 2015-11-28 04:23:43 Minutia Capture history for archive 1TB_FOTO_ARCHIVIO_PORTABLE_HD.alias appears to have been deleted
...

That's a bug. In this case, QRecall discovered a status file for a version of 1TB_FOTO_ARCHIVIO_PORTABLE_HD that had at some point been deleted, or overwritten. It's supposed to log this message once and delete the (orphaned) status file. The delete step was broken.

QRecall 2015-11-28 09:12:40 Warning Report generator exited with status 15; please report this to the developer

Odd, and I'll look into it, but I'm not overly concerned about it. Apparently an problem occurred trying to sample some of QRecall's background processes in preparation of uploading a diagnostic report. But the report came through, and was intact, so I'm guessing this was a transient problem.

As I in part noted with the report, here are things that now don't quite look right:

1. Yes I clicked in the Status Window to have QR 'forget' the 1TB_FOTO_ARCH - xx history since that archive no longer exists and I have created a new archive for that external HD. But having the log show the message repeatedly like this?

Unfortunate coincidence, and unrelated to the real problem.

2. Archive 1Tb-xx shows in the status window as amber but I can't see why. No pending actions; a Verify went to completion today. Still amber.

Expand the status for 1Tb-xx and see which status is amber?capture, verify, or size?and what the status message is. The status indicators change for a variety of reasons, many unrelated to the health of the archive.

3. Archive MILLIE-xx is in the green on the Status Window but opening the archive shows that layer 0 is damaged:
Layer 0 -- Date UNKNOWN -- Size UNKNOWN -- Items DAMAGED.
---->> How should I regard this? What should I do to recover the data in this layer?

A damaged layer indicates that one or more items in that layer was, at some point in time, lost during a repair. It's a concern only if you recall items from that layer, as some items may be missing or incomplete. Eventually, the missing data will be replaced when that layer is merged with subsequent layers, and the "damaged" indicator will go away.

Your other issue, regarding not being able to repair your other archive, has also been fixed in 2.0.0b25, which should appear on the server shortly. Update to the latest version and try that repair again.

These are pretty minor issues, and certainly don't warrant the reinstallation of any software.

The CrashPlan.app error is a curious one, but probably inconsequential. The error indicates that during the recall QRecall was trying to restore the last modified date, creation date, last attributes change date, and the permissions (access mask) for the app bundle folder. For some reason, the operation was prohibited by the operation system. It might be the case that the Crash Plan app has a special ACL that prohibits changing one or more of these attributes.

But since we're talking about the app bundle folder, as long as it's readable that's just about all that matters.

The second error is equally mysterious, but also doesn't really matter. During the restore, QRecall determined that there were ACLs attached to the Documents folder, but the last time it captured the Documents folder there were none, so it tried to delete the ACLs. Again, this was prohibited by the OS for some reason. This might be a new behavior in 10.11 due to SIP. I'll have to look into it.

Overall, pretty trivial issues for a full restore of El Capitan.

Bob Tyson wrote:James, the capture of a new archive from the 1tb external HD is complete and I have sent the report.

Thanks!

Here is an oddity. I just now tried to copy a folder from my Mac to that 1tb external disk and could not do so without entering my admin password for the Mac's system. I did so, made the copy to the external HD, and then checked the latter's permissions. They were read-only for 'everyone' and read-write for 'system'. I changed the 'everyone' permission to read-write. But I am noticing various disks and folders seem to have had their permissions changed (?) to 'read-only ' for 'everyone'.

A Finder copy should not (normally) change the ownership of items, unless you're copying to/from a volume that has the "Ignore ownership and permissions" property set. But even then, the ownership of items on those volumes will always be "you" (the logged in user).

Another thing to be aware of is that creating, moving, renaming, or deleting an item also involves the ownership of the folder that contains (or will contain) that item. In other words, creating a file is considered a "change" to the folder that contains the new file, and you must have write permission for that folder to accomplish that.

And in the "really obscure" department, there are special attributes (call Access Control Lists, or ACLs) that can be set on a folder that to do things like automatically set the ownership of items added to that folder and such, but it's unlikely that's the problem here.

Bob,

I'd still appreciate a diagnostic report. I'm interested in all of the details that don't appear in the log window.

Bob Tyson wrote:James, here are notes I added to the report. And **sigh** it appears to be just that one new archive. At least the other, which I had captured successfully afterwards, is now being captured again. (Sorry -- I get befuddled with terminology here: 'Capture' meaning in this case, capturing to a pre-existing archive, from an HD previously backed up to that archive; distinct from 'Recapture'?? Or what am I missing, still after 'x' years?

It doesn't matter what you call it—capture, recapture, backup, vacuuming, ...—under the hood it's all the same thing.

At any rate it does appear the large, new archive is the culprit.

Yes and no, at least from the information I have. Most of your QRecall app crashes appear to be related to a race condition in the browser code. One thread is trying to load a large folder in the background at the same moment that you're closing the window/archive. That's what's causing the crash. So it's related to the new archive only in the fact that the browser was open to a particularly large folder when you closed the window. These are all very similar to crashes other users have reported, and all are under investigation.

EDIT - I attempted 'REPAIR' from the QR FILE menu. After confirming the volume OK I chose REPAIR without checking any other box. As soon as the repair began it failed - here are the log entries:

Action 2015-11-26 09:02:30 ------- Repair 1TB_FOTO_ARCHIVIO_PORTABLE_HD
Action 2015-11-26 09:02:30 archive: /Volumes/TO BACK 03 3T/1TB_FOTO_ARCHIVIO_PORTABLE_HD.quanta
Action 2015-11-26 09:02:30 Failure Failed
Action 2015-11-26 09:02:30 problem setting EOF
Action 2015-11-26 09:02:30 Failure Repair failed
Action 2015-11-26 09:02:30 A network or disk error was encountered.
Action 2015-11-26 09:02:30 ------- Repair incomplete (00:00)

Now that sounds serious.

But it got worse. As a double-check I tried to repair the second archive, the one I successfully updated and that seems ok. The repair started ok, and after a couple of minutes I clicked 'STOP'. After perhaps another minute a message came forth saying the archive index was damaged needed to be reindexed. This seems odd to me, but I am presently reindexing that archive, following the prompts.

It's not surprising to me at all. When you start a repair (or reindex), QRecall blows away the existing index files and starts rebuilding them from scratch. The archive will be unusable until the repair has finished. Canceling it pretty much guarantees that your archive will remain "damaged".

During the backup from the external HD to the new archive I trashed several large files from that HD. QR completed the capture, then went into a verify and reported problems with both the backup and the verify.

I see from the log included in the diagnostic report where the new archive was created, you captured new files to it, deleted a few, and then performed a capture on another, existing, archive.

From the logs, all of that went very well. There were some warnings in the first capture about missing items because (I suspect) you were either moving or trashing some OS installer apps while the capture was in progress. It's a non-issue unless you were planning on recalling those, partially captured, items.

But that's all I've got. I assume the repair and some of the other issues you mention occurred after you sent the report. Please send another so I can review what is/was going on with the repair actions.

Bob,

Yikes! I thought most of those issues had been dealt with in b11.

Send a diagnostic report and I'll take a look. In the report, let me know if it's just the one archive or all archives the cause the crash.

Bryan Derman has just released rev C of his optical media backup scripts for QRecall 1.x.

Rev C adds some important new features, including support for multiple backup sets (for rotating off-site storage) and the ability to reconstruct a set following changes to the archive (e.g. a repair), intelligently rewriting the minimum set of discs required.

You can read more about the changes and download the latest scripts from his website: Off-Site and Archival Storage for QRecall Backups