Bruce Giles wrote:OK, so if I want to get rid of those damaged ("Unknown Unknown Repaired") layers I need to keep merging them with subsequent layers until I eventually hit a layer that recaptures everything that was still missing, and at that point, the damaged layer disappears.

That is correct.

You REALLY don't want to see what the System Log file in the Console app does to me.

I feel your pain ... especially when I look at my server.

With regard to the verify errors discussed in this topic, when the Verify command reports an error and you choose to repair it, you get the option of verifying or repairing your volume first, which is good. But no matter whether you choose verify volume or repair volume, the top of the sheet says "Repairing volume". That confused me a little, when I was sure I had clicked the Verify button, not the Repair button.

That's static text (it doesn't change, except for the name of the archive). I've changed it to "Check volume containing ...".

Second, after I got past the volume verify/repair, when the Repair Archive sheet comes up, the only option not grayed-out was "Reindex only".

When you click on the handy "repair" button in a error message or dialog, QRecall pre-configures the repair options based on what it thinks the problem is. For problems with the index files, it assumes a reindex is all that's necessary. It the error is in the primary data file, the options are pre-configured for a full repair.

In your case, the first problem discovered was in an index file and QRecall assumed a reindex would fix it. It did not. The reindex discovered a problem in the primary data file, so the second time the repair options were pre-configured for a full repair. But you could have changed it to a repair on the first go.

Once in the dialog, you can reconfigure the options to whatever you want. Just be aware that the options are highly interdependent. The "Reindex only" option is incompatible with all of the other options, so as long as "Reindex only" is checked, all of the other options are disabled. Uncheck "Reindex only" and you'll have the opportunity to select a different set of options. The "Use auto-repair information" and "Copy recovered content to new archive" options are also mutually exclusive; selecting one disables the other. And once you've picked any other option, the "Reindex only" option will be disable, because, you know, mutually exclusive.

David Ramsey wrote:If I invoke QRecall by opening the archive file directly, QRecall comes up and immediately crashes.

Try sending a diagnostic report, which will include your crash logs. If you can't send a report, just email a copy of your QRecall* crash log files in ~/Library/Logs/DiagnosticReports/ and your log files in ~/Library/Logs/QRecall/ to support@qrecall.com.

(If they are large, you may want to compress them.)

Bruce Giles wrote:So, it seems that and Unknown layer can't be eliminated by merging, but if you try, it appears to "eat" the subsequent good layer.

You can "fix" a layer by merging, but only under specific circumstances.

When merging with a damaged or incomplete layer, the later (good, complete) layer must have captured anything marked as damaged or incomplete in the earlier layer for the merged layer to be considered complete again.

For example, on Monday you capture your Documents, Music, and Pictures folders. On Tuesday you recapture your Music and Documents folders.

On Wednesday, a media failure damages your archive requiring it to be repaired. During the repair, the Pictures and Music folder in Monday's layer are marked as damaged because data from one or more files in those folders was lost.

You now have two layers. Monday's layer is marked as "damaged".

If you merge these two layers, the merged layer will still be marked as "damaged" because Tuesday's layer did not recapture the Pictures folder from Monday. It did recapture the Music folder, so that folder is no longer damaged in the layer, but it inherits the damaged Pictures folder, so the layer is still "damaged".

If you recapture the Pictures folder in a new layer, and merge that with the previously merged layer, the new layer will be complete.

(Note that when a capture action sees that the latest layer(s) are damaged or incomplete, it automatically ignores filesystem change history and forces the recapture of all files and folders marked as damage or incomplete. This guarantees that if you then merge the newly captured layer with the previous one(s), the resulting layer will be complete.)

And this actually gives me an idea for a new feature. I think we need a "damage" report or search mode that will show you just those directories or files that have been damaged.

Bruce Giles wrote:For the next ten minutes or so, nothing happened, except that the window said something about waiting for permission to open the archive.

QRecall 2.0 uses a new method for arbitrating concurrent access to an archive. Inside the archive package are some invisible semaphore files (.lock, .share and .semiphore). These allow QRecall to play nice on filesystems that don't support the read-shared/write-exclusive access modes, and on networked volumes, NAS devices, and so on.

An outstanding bug, that I'm still investigating, is that the .lock file doesn't get "unlocked" when it should. This causes QRecall to think that a remote process is using, or updating, the archive. QRecall performs a series of tests, which take about 9-10 minutes, to ensure that no other process is modifying the archive. Once it's convinced that the semaphore files are stale, it resets them, acquires the access it needs, and proceeds to perform the action.

If you haven't done so, please send a diagnostic report. I'm still looking for combinations of actions the leave the .lock file locked.

Thank you, everyone, for patiently reporting this problem.

I think I finally figured it out. QRecall 2.0.0b18 is out and contains the fix.

I kept thinking that it was a problem with the logic that finds and compares volume identifiers. As it turns out, I was going in the wrong direction. The issue was that the volume identifier was doing too good of a job, identifying mountpoint directories as volumes when they should have been treated as sub-directories. The details are explained in the release notes.

Happy capturing!

James Bucanek wrote:

Ming-Li Wang wrote:After some digging just now I found my user profile folder was the culprit. Now, remember I said I had a dedicated partition for my documents which is mounted at boot at ~/Documents? I unmounted the partition and tried again and sure enough the test archive verified without error.

I can't imagine any reason why your environment would make any difference to the verify. In theory, the verify simply compares the structure defined by the individual directory records with the index of the directory structure. It shouldn't matter what your computer environment is, and should succeed even if ran on a completely different machine.

Having said that, I should also say that I should never say "never".

(It's lucky I added that caveat.)

I do have one more question: did you unmount the volume (mount point) at ~/Documents before you made the capture, or before you performed the verify?

Because I'm now beginning to suspect that the former could make a difference...

Ming-Li Wang wrote:After some digging just now I found my user profile folder was the culprit. Now, remember I said I had a dedicated partition for my documents which is mounted at boot at ~/Documents? I unmounted the partition and tried again and sure enough the test archive verified without error.

I can't imagine any reason why your environment would make any difference to the verify. In theory, the verify simply compares the structure defined by the individual directory records with the index of the directory structure. It shouldn't matter what your computer environment is, and should succeed even if ran on a completely different machine.

Having said that, I should also say that I should never say "never".

They only started since beta 16, or 15 at the earliest.

This I agree with. I strongly suspect that a subtle change in how volume identifiers and mount points are dealt with is the root cause of this issue. I've been poring over the code change logs looking for a cause, but have yet to identify it.

I have a new beta that I'll probably release today. It includes some code to log more information about these verify errors, in the hopes that I can get more examples to look at. If you encounter this issue with 2.0.0b18, please send a diagnostic report and (if practical) stop using that archive. I may have some diagnostic commands for you to run on it.

The search continues in ernest.

Please be aware that there's still a bug in the current QRecall 2.0 beta that may cause the verify command to report that the archive's catalog tree is invalid or that there are "unreachable" directories.

This issue is still being investigated, but early exploration would indicate that this is an erroneous error; the archive is not damaged and does not need to be repaired.

For the time being, please ignore this error. That is, you don't need to repair your archive if the only action complaining is the verify, and it is reporting one of these two failures.

But do please send a diagnostic report should it happen to you. In the comments, please indicate if the archive was created with QRecall version 2.0 or if it is an older archive that you converted to 2.0.

Apple released OS X Server 5.0.4 a few days ago, and the upgrade inadvertently broke the *.qrecall.com website.

Apple's new server includes an integrated proxy service that intercepts and pre-processes all of the requests handled by the Apache web server. Unfortunately, in the process each request gets modified. A lot of pages, particularly the secure pages used to log in and access your account (and buy stuff!), depend on information in this request. This change completely flummoxed the logic the ensures you only access account information via HTTPS, which made it impossible to log in.

Anyway, it should be working again now. If you discover any issues logging in or accessing your account, please drop a note to support@qrecall.com.

All layers earlier than the two 1-year layers defined will be merged into a single layer, but that will be considerably more than 2 years ago.

When you run the merge today (October 14, 2015), here's how QRecall will calculate your merge.

You're keeping the most recent 7 days, so any and all layers captured after Oct 7 will be left as they are.

In the next tier, you have 90 day layers. QRecall starts at Oct 7 and looks backwards for the first full day, which is Oct 6. All layers captured on Oct 6 will be merged into a single layer. It repeats this for the remaining 89 day layers:

Day 90: Oct 6, 2015
Day 89: Oct 5, 2015
Day 88: Oct 4, 2015
...
Day 1: July 9, 2015

Your next tier has 52 week layers. Starting with July 9, 2015 QRecall looks for the nearest full week, which ended on Saturday July 4, 2015. All layers captured between Sunday June 28, 2015 and Saturday July 4, 2015 will be merged into a single layer. This repeats for the previous 51 weeks:

Week 52: Sat June 28, 2015 - Sun July 5, 2015
Week 51: Sat June 21, 2015 - Sun June 27, 2015
Week 50: Sat June 14, 2015 - Sun June 20, 2015
...
Week 1: Sat June 29, 2014 - Sun July 5, 2014

The next tier keeps 12 month layers. Again, started at the beginning of the last tier (June 29, 2014), QRecall searches backwards for the first full month, which is May, 2014.

Month 12: May, 2014
Month 11: April, 2014
Month 10: March, 2014
...
Month 1: June, 2013

Finally, you have 2 year layers. These will start on the previous full year before June, 2013:

Year 2: 2012
Year 1: 2011

Finally, all layers captured before January 1, 2011 will be merged into a single layer.

Every thing else: 2006(*) - 2010

I hope that helps.

(*) Why 2006? Because that was the first public release of QRecall. Obviously, you can't have any layers before that.

If anyone is using, or interested in, Bryan Derman's scripts for backing up QRecall archives onto optical media, you'll be happy to know that there's a new version available: Off-Site and Archival Storage for QRecall Backups

Bryan is now using QRecall in combination with his scripts to secure the data on several mission critical systems. If you have similar needs, please give these (free!) scripts a try and let Bryan know what you think.

My advice would be to wait until I fix this bug

I'm pretty confident that there's no data loss, just migration pains.

Ming-Li Wang wrote:The latter. The date stamp for layer 0 is 2015-01-31, so definitely 1.2.x.

This seems to be the problem. I've identified several issues with archives that were originally created with 1.2 and later updated to 2.0.

I'm still investigating, but this problem seems to be related to how volumes are identified in 2.0 vs. 1.x. This disagreement causes newly captured items to migrate into a new directory hierarchy that, in turn, trips a number of internal consistency checks. In your case, a delta directory record (a database record that records changes in a previously captured folder) can only refer to a another directory record that is both (a) in an earlier layer and (b) at the exact same location in the volume/folder hierarchy. The shifting volume identifiers cause the second test to fail, making the inherited directory record "unreachable."

The verify may also report "layer catalog tree" errors, which basically means the folder structure in the index and the folder structure the verify thinks the archive should have aren't identical.

I have seen this with another archive.

Important question: Is this an archive created with the 2.0 beta, or is this a 1.2.x archive that you started using with 2.0?

Just FYI, I think I've got a solution in the works.

There were about four different, and subtle, problems that intersected the code which deals with users who have relocated their home folders to a different volume. I have definitely fixed a number of bugs for those folks. Now I just have to determine if I've also fixed the related issue for users with regular home folders.

I should have a new version out by tomorrow.