David Ramsey wrote:1. My backup currently has 57 layers. Is that an unusually high number? Should I merge more often?

The number of layers will have some impact on performance, but doesn't fundamentally change how files are restored. As far as QRecall is concerned, there's no difference between 2 layers and 200 layers. And for the record, I have archives with over 500 layers that restore just fine.

2. What's going on when the barber pole is animating?

There's a quirk that affects very large restore actions. There's a separate process that estimates the amount of data that's going to be restored. It takes awhile to generate that estimate, which is why you initially see the barber poll. The estimate process, however, can underestimate the amount of data to restore. Once the restore has restored the amount of data in the estimate, it switches back to the barber poll because at this point it doesn't know how much data remains to restore.

3. Any ideas on why a whole-volume restore didn't work?

I don't know. In theory, you shouldn't have any problem restoring a bootable OS X volume. Here's some things to look out for:

The "Show Deleted Items" view option was not turned on.

After formatting the new drive, and before restoring, make sure the "Ignore ownership and permission" option was not checked.

I assume that both the capture and restore were authorized to use administrative privileges.

I assume you restored the volume by holding down the option key and choosing Archive > Restore Volume To... (or by dragging the archive volume icon, dropping it into the restore volume, and choosing the "Replace Volume" option when the dialog popped up).

Did you try blessing the volume after restoring it?

Charles Watts-Jones wrote:While back-ups run at an acceptable speed, verify operations are terribly slow at around 6.5 MB/min.

6.5 MB/min is dreadfully slow. Are you sure you don't mean 6.5MB/sec? That's about what I would expect from WiFi.

Verifies are I/O bound. If you have 100Mb ethernet, connecting via ethernet will give you about 10MB/s. If you have Gb ethernet, you should get about 80-100MB/s.

I don't know what your topology is, but consider having a different computer verify the archive. I have several laptops and an iMac that all capture via WiFi, but I use the server that's connected directly to the shared RAID to perform all of the weekly verify and merge actions. Everything is fast.

Charles Watts-Jones wrote:When using an external drive QRecall was able to back-up when I was logged out at night. Is it possible to arrange this with a NAS (which runs 24/7)? Or must I leave myself logged in 24/7 too?

It depends on how the volume mounts. If the volume mounts as a physical hard drive, QRecall will mount it before starting an action. If the mounts as a remote/network volume, then that requires an account and password, which requires the keychain, which requires that you be logged in and your keychain unlocked when the action runs.

A little experimentation should tell you which method works.

Testing the NAS I made a new archive on the NAS. I never used it and have deleted it. Status Window remembers it and lights its red button to show that the archive has never been used. How do I get Status Window to 'forget' that archive?

Control+click/right-click on that item in the status window and choose Forget from the pop-up menu.

Ralph Strauch wrote:I haven't been compacting because I was under the impression that qrecall would use empty space in the archive as it became available, ...

QRecall does (normally) reused the empty space in an archive. Over time, however, these bits of unused space become more and more fragmented. Eventually, QRecall must keep track of hundred of thousands, if not millions, of tiny, empty, records. It's this overhead that you're encountering.

Compacting consolidates this empty space by relocating active records so they are adjacent. It also rearranges records so that like records are closer together, which makes accessing the archive more efficient.

Is there any simple way tell how much empty space there is in an archive?

Open an archive and make sure nothing in the item (or layer) browser is selected. Open the Info window and it will show you a summary for the entire archive, including the amount of unused space (if it's been recently calculated).

But the problem here is not the amount of unused space, it's the amount of fragmentation, and the browser stats won't tell you that. You can infer the amount of fragmentation by looking at the size of the fill.index file inside the archive.

Ralph,

Thanks for the samples and the diagnostic reports.

In the first report and sample you sent, something definitely seems amiss, but I'm at a loss to explain why. The helper process appears to be stuck trying to access the record index file, but there's no obvious reason why it should be having trouble doing that.

In the second sample, QRecall is actually hard at work. It would appear from the log that your archive has accumulated a lot of empty space. QRecall keeps track of the unused portions of your archive and reuses it. As the number of unused areas increases, so does QRecall's overhead. It must periodically stop and clear the unused space that has accumulated?QRecall deliberately erases all deleted data from an archive. This is what it was doing when you made the second capture. This maintenance can occur at any time during a capture and it would appear that the capture is "hung up."

This also exacerbates another problem. If you terminate (force quit) a QRecall process, then next action has to auto-repair the archive before it can proceed. Among the auto-repair tasks is to re-erase all of the empty space in the archive. In your logs, I'm seeing that QRecall it taking upwards of 20-30 minutes to accomplish this, which I suspect means there's a lot of empty space in your archive.

I would suggest reindexing the archive first: choose the File > Repair Archive command, open the archive, and then check the Use auto-repair and Reindex only options.

When the reindex is finished, open the archive and compact it (Archive > Compact). This will consolidate and recover all of the empty space in the archive and optimize the record order for better performance.

I highly recommend that you create a compact action and schedule it to run occasionally (say, once a month) for optimal performance.

Ralph,

Thanks for the diagnostic report. But as you might imagine, the log doesn't help much in determining why a process has become stuck.

If this happens again, it would help greatly if you could capture a sample of the process. A capture process runs as root, so this must be done through the command-line. Launch Activity Monitor and get the process ID (PID) of the QRecallHelper process that's running as root (make sure you have the All Processes filter set). Substitute that PID number for "PID" in the following command:

sudo sample PID 20 -file ~/Desktop/QRecallHelper.sample.txt

It will prompt you for your admin password, and then sample the process for 20 seconds. When it's done, send me the QRecallHelper.sample.txt file that's now on your desktop.

Jonathan Edwards wrote:It's good to know I'm not the only one with the keyboard focus issue in Column View. I thought I'd provide a screenshot of this issue all the same and in doing so have discovered what looks like a second bug:

"Folder 2B" is not being shown at all in this view. It was captured and resides in Folder 2, as shown in both the equivalent Icon View and List View screenshots.

That's strange. I'll look into it. A lot of the column view code is getting rewritten for the next version, so I'll see if something like this is still happening.

Also attached are two Layers View screenshots, as requested, showing the graphics corruption.

That's even stranger. It looks like the view is not being completely redrawn. Does the situation change/improve if you resize the window, particularly if you make the window smaller?

Also, please send a diagnostic report from QRecall. This will include a snapshot of your hardware configuration (particularly, your GPU).

Jonathan Edwards wrote:I see that changing the advanced setting QRCompactErasesExpiredDeletedItems to false prevents "deleted" items outside of the "Keep deleted items" setting from being permanently deleted by a Compact action, which is great.

Is there an equivalent setting for the Merge action please? Thus, keeping all "deleted" items whilst retaining the additional benefits of the Merge and Compact actions. A hidden alternative to a "Keep deleted items forever" option in the archive settings perhaps?

There is no "keep forever" option but, as I posted previously, you can set the keep deleted items time to 999 months, which is effectively "forever." Deleted items will be retained for 80+ years.

If you've done this, the QRCompactErasesExpiredDeletedItems setting will have no effect, because you won't have any deleted items that have expired.

On a separate note, there appears to be a file browser bug when using the arrow keys in column view:

That's a known bug.

I wonder if this is due to an incompatibility with the Macbook Pro's Retina display?

Nope. It's a quirk of how QRecall handles keyboard focus.

That may also account for the distorted 3D Layers view I'm seeing. Changing the Automatic graphics switching preference in Energy Saver has no affect.

This sounds like an unrelated issue. If you could send me some screenshots, that would be very helpful.

This is still invaluable software, thanks!

That's great to hear!

My only comment is to remember the immortal words of Roy, from The IT Crowd:

"Have you tried turning it off and back on again?"

Charles Watts-Jones wrote:Have run the complete set from the Actions window. All OK. Will report again after tommorrow morning's schedule.

Then I would expect it to work. If it doesn't then something else (besides the archive location) is the issue.

When you run an action from the activity window, it actually goes through the scheduler. QRecall (the app) tells the scheduler process to schedule that action to run "now." So there's almost no difference in running an action manually and scheduling it to run at a particular time.

Charles Watts-Jones wrote:As a test I ran the Capture action, it ran flawlessly.

Out of curiosity, did you run the capture action from the actions window or from the File menu?

Using the menu File > Open Recent QRecall opened the archive in its new location.

The Recent Items menu is managed by OS X. How is locates archives is completely different, and independent, of how QRecall locates them.

I must be missing something but what?

I can't think of what. I would suggest that you need to update the location of your archive in your actions, but you seem to have done that already.

The test is to open your actions window and look for actions that are in grey or have a warning icon (see attachment). QRecall's actions window uses the information from the action document to locate the archive, exactly the way the action does when it runs.

If the archive appears grey or has a warning, there are still locations that need to be updated. Open the archive document, select the archive again, and save it.

It they all say they are ready to run, try running them from the action menu (assuming you didn't try that before). If they run successfully, and yet they don't run when scheduled, then something is clearly off. Please send a diagnostic report and we'll investigate.

Hanno,

I downloaded and tested Boxcryptor Classic on a MacPro (10.8.5). Here's what I found.

The Boxcryptor system creates a bundle (Boxcryptor.bc) in your Dropbox/cloud/whatever folder. The package contains the encrypted versions of your files. As you already know, QRecall can capture these just fine.

The Boxcryptor Classic app creates a virtual files system that mirrors those files in an decrypted form. The problem is that the volume doesn't behave like a regular volume.

When I captured the volume, QRecall immediately reports that the volume fails to report its extended attributes (listxattr() returned error 2, which is "no such file or directory"). This pattern gets repeated when QRecall attempts to capture individual files in the volume (file not found and folder not found).

If I had to guess, I'd say that Boxcryptor's implementation of their filesystem fails to support extended attributes and/or the fsIDs used by the Carbon filesystem API to identify objects on a volume. That could explain why QRecall's attempt to open a file or folder return "not found". The next version of QRecall might have better luck, as we're switching to the BSD filesystem API, which tend to be friendlier with foreign filesystems.

Kevin Crawley wrote:The Actions window only shows my new schedule actions.

That's the most definitive indication of what actions you have scheduled. I suspect that your status window (which is run by the QRecallMonitor app) is simply out of sync, possibly related to the problems you've had with your other actions.

Simple solution: Restart your computer.

If you want to try this piecemeal, use the Activity Monitor app. Start by entering "QRecall" into the filter field.

If you don't currently have any QRecall actions running, then there should not be any QRecallHelper processes running. If your find any, Force Quit (kill) them. (If the process is stuck on an I/O error, however, killing it won't work as the process is stuck in the kernel. You will have to hard restart your computer.)

Send a Quit message to the QRecallScheduler process. Then send a Quit message to the QRecallHelper process. Both should stop and then be immediately restarted (they'll have new process IDs when they reappear). If that doesn't happen then it's possible these processes are stuck in the kernel too, and you'll still have to restart your system. (Low-level failures really gum up the works.)

Kevin Crawley wrote:kernel en0: can't handle af8

Googling it brings up nothing. Could it be yours?

Not a QRecall message. (Any QRecall message in the console will have a QRecall process name associated with it.)

en0 is the first ethernet adaptor. Probably something to do with IP traffic.

Kevin,

I regret to say it, but I suspect that your hard drive is dying. There may be I/O errors in QRecall's log, and you're welcome to send a diagnostic report, but from what you describe it would appear that your hard drive is encountering read failures when trying to read certain regions of the platter—specifically, those that the QRecall archive occupies.

When I test it with the Disk Utility or with Disk Warrior, they tell me the disk is okay ...

Disk Utility, Disk Warrior, and almost every other disk utility out there, only validates the directory structure of the volume. They do not perform a comprehensive read/write test to confirm that all sectors of the drive are usable. You could lose a third of your drive, and Disk Utility will still report that it's fine. Any utility that reads the entire surface would require hours (and hours) to run. This is why QRecall's verify action takes as long as it does.

If you're interested in testing this theory, it's pretty easy from Terminal. You can run a simple command to read all of the data in the archive. If it's successful, it will finish after an hour or so. If there are I/O failures, the command should stop and spit out an error.

cat /Volumes/My3TBExternalVolName/pathtoarchive/ArchiveName.quanta/* > /dev/null && echo "No read problems"

Be patient. Even if it works, it will take a long time to finish. If the drive is encountering I/O failures, it will likely enter a retry/recalibrate cycle where it attempts to realign the heads and try again. Drives that are doing this run about 1,000 times slower than normal, so even if it hits an I/O error it could be many minutes before it's reported. You may hear the drive heads seek and clatter during this process. You can monitor the progress using Activity Monitor (switch to the Disk Activity tab) to see if it's still reading the drive or if the command has stalled. (Press Control-C in the Terminal window to kill the command.)

If you verify that it's I/O errors, you can attempt to recover the archive. You'll need a second volume with enough room to copy it. Choose the Repair Archive... command. Uncheck the "Use auto-repair information" option. (This is probably where your repair is getting stuck now.) Check the "Copy recovered content to new archive" and "Recover lost files" option.

This configuration will doggedly try to read every record in the original archive. Every record that can be read will be copied to the new archive. QRecall will then assemble the salvageable data into a usable archive. The "Recover lost files" option will collect orphaned files—files that don't belong to a folder because all of the folder information was lost—into a special "Recovered files" volume.

If the drive has a lot of damaged areas, the repair could run for days before it completes. So again, you'll need to be patient.