I've had this intermittent problem crop up a few times, where qrecall hangs up when it starts to capture a backup. The monitor window opens the archive and shows "Capturing .config" and just sits there. It doesn't respond to cancel requests, although the log shows that they have been received.I have to force quit qrecall and qrecall helper, and sometimes even restart the computer, to get it back on track. It usually works the second time, though.

Today it was a bit different. Instead of hanging immediately when capture started, it captured 14.6mb and then hung, "Capturing Adobe Reader." Force quitting all the qrecall components and even logging out didn't help. Qrecall would start again showing that capture still running. I finally restarted the computer and that let me restart qrecall and the backup normally, but then it hung again while "Capturing .config" and is still sitting there after almost an hour.

The computer is a 2011 MacBook Pro running 10.8.5, backing up over wifi to a drive attached to an iMac. The iMac backups are fine, and have never exhibited this problem.

I'm sending a report from the MBP.

Ralph

Ralph,

Thanks for the diagnostic report. But as you might imagine, the log doesn't help much in determining why a process has become stuck.

If this happens again, it would help greatly if you could capture a sample of the process. A capture process runs as root, so this must be done through the command-line. Launch Activity Monitor and get the process ID (PID) of the QRecallHelper process that's running as root (make sure you have the All Processes filter set). Substitute that PID number for "PID" in the following command:

sudo sample PID 20 -file ~/Desktop/QRecallHelper.sample.txt

It will prompt you for your admin password, and then sample the process for 20 seconds. When it's done, send me the QRecallHelper.sample.txt file that's now on your desktop.

James,

Once it starts the process seems to keep running until I force-quit QR Helper in Activity Monitor, or restart the computer, so the attached sample if from a process that's been running for 5 hours.

Activity Monitor does not show a QR Helper process running as root. Two versions are running, both as ralph. The one I sampled is taking up 0.1-0.3% of the CPU, and using about 1gb of real memory and slightly more virtual memory, with 7 threads. The other one has 3 threads, uses no CPU, and much smaller amounts of memory.

Let me know if you'd like to me restart a fresh process and sample that.

Ralph

James,

Here's a sample from another hang-up, after running for about 7 minutes. Like the earlier on, it is one of two instances of QR Helper belonging to ralph, the one using more CPU and memory resources. This was using slightly more than 100% of CPU (on a 4 processor Mac) when I ran this, and is now down to 3%.

Let me know if you need anything else.

Ralph

Ralph,

Thanks for the samples and the diagnostic reports.

In the first report and sample you sent, something definitely seems amiss, but I'm at a loss to explain why. The helper process appears to be stuck trying to access the record index file, but there's no obvious reason why it should be having trouble doing that.

In the second sample, QRecall is actually hard at work. It would appear from the log that your archive has accumulated a lot of empty space. QRecall keeps track of the unused portions of your archive and reuses it. As the number of unused areas increases, so does QRecall's overhead. It must periodically stop and clear the unused space that has accumulated?QRecall deliberately erases all deleted data from an archive. This is what it was doing when you made the second capture. This maintenance can occur at any time during a capture and it would appear that the capture is "hung up."

This also exacerbates another problem. If you terminate (force quit) a QRecall process, then next action has to auto-repair the archive before it can proceed. Among the auto-repair tasks is to re-erase all of the empty space in the archive. In your logs, I'm seeing that QRecall it taking upwards of 20-30 minutes to accomplish this, which I suspect means there's a lot of empty space in your archive.

I would suggest reindexing the archive first: choose the File > Repair Archive command, open the archive, and then check the Use auto-repair and Reindex only options.

When the reindex is finished, open the archive and compact it (Archive > Compact). This will consolidate and recover all of the empty space in the archive and optimize the record order for better performance.

I highly recommend that you create a compact action and schedule it to run occasionally (say, once a month) for optimal performance.

James Bucanek wrote:
In the second sample, QRecall is actually hard at work. It would appear from the log that your archive has accumulated a lot of empty space. QRecall keeps track of the unused portions of your archive and reuses it. As the number of unused areas increases, so does QRecall's overhead. It must periodically stop and clear the unused space that has accumulated?QRecall deliberately erases all deleted data from an archive. This is what it was doing when you made the second capture. This maintenance can occur at any time during a capture and it would appear that the capture is "hung up."

This also exacerbates another problem. If you terminate (force quit) a QRecall process, then next action has to auto-repair the archive before it can proceed. Among the auto-repair tasks is to re-erase all of the empty space in the archive. In your logs, I'm seeing that QRecall it taking upwards of 20-30 minutes to accomplish this, which I suspect means there's a lot of empty space in your archive.

Thanks for these explanations. One of the reasons I've force-quit QR as often as I have it that it often seems unresponsive to repeated cancel requests without giving any indication that it's still doing anything. If it's possible to give some kind of indication that the app is still productively working that would be useful.

Another example: I started a Compact action (running on the iMac where the drive is mounted, after turn off file sharing, rather than over the network from the MBP). I then realized that I hadn't merged layers for a while and it would make more sense to that before the Compact rather than after it. So I immediately tried to stop the Compact, and got no obvious response -- QR appeared to be in a hung state. In light of your explanation (above) though, rather than force-quitting, though, I just let it sit and went away for a while. It took over an hour for the Compact action to clearly terminate. I then ran my Merge action and followed it with Compact again, which is what it's doing now. If it hadn't been for your explanation I would have given up and force-quit the Compact long before it quit on its own.

Both QR and the Monitor window on the MBP continued to show the backup action as running, in spite of several cancel requests, quitting and restarting QR several times, the fact that the archive was no longer accessible. I had to restart the computer to reset QR to its proper state. I'm reporting these detail for your information, and don't need a response right now. Tomorrow after the Compact is finished I'll try another backup and I'll let you know what happens.

BTW, through all these problems with my MBP backups, the daily backups of the iMac have run without any problems.

Ralph

Compacting my archive took 22 hours and recovered 64gb from an 875gb archive. Both computers have since backed up with no problems.

James Bucanek wrote:I highly recommend that you create a compact action and schedule it to run occasionally (say, once a month) for optimal performance.

I haven't been compacting because I was under the impression that qrecall would use empty space in the archive as it became available, but I guess I should change that behavior. Is there any simple way tell how much empty space there is in an archive?

Ralph

Ralph Strauch wrote:I haven't been compacting because I was under the impression that qrecall would use empty space in the archive as it became available, ...

QRecall does (normally) reused the empty space in an archive. Over time, however, these bits of unused space become more and more fragmented. Eventually, QRecall must keep track of hundred of thousands, if not millions, of tiny, empty, records. It's this overhead that you're encountering.

Compacting consolidates this empty space by relocating active records so they are adjacent. It also rearranges records so that like records are closer together, which makes accessing the archive more efficient.

Is there any simple way tell how much empty space there is in an archive?

Open an archive and make sure nothing in the item (or layer) browser is selected. Open the Info window and it will show you a summary for the entire archive, including the amount of unused space (if it's been recently calculated).

But the problem here is not the amount of unused space, it's the amount of fragmentation, and the browser stats won't tell you that. You can infer the amount of fragmentation by looking at the size of the fill.index file inside the archive.

OK, Thanks. I'll set up a nightly compact action and watch it for a while.

Ralph

I had a problem again today with Qrecall hanging up during a capture. It's all cleared up and not currently a problem, but I'm passing on this history FYI in case it's of value to you.

This time it happened on my wife's iMac, during a backup that startedat 4:15am, after the completion of Merge and Verify actions. When I looked at the computer some time after 10am it showed 27 items captured, 273mb, 80% duplicate, after running for more than 6 hours. These backups usually take much less than an hour. I left it and came back again at 12:30, to find the status unchanged. At that point I sent several cancel requests, to which qrecall did not respond. Earlier in this thread you said that qrecall could take some time to quit normally, so I left it alone to see what would happen.

When I came back to the computer at 10pm, the status was still the same. I terminated the process by quitting (not force-quitting) qrecall helper from Activity Monitor. It terminated immediately, and logged the completion of the interrupted backup. I then immediately restarted the backup, which ran normally and completed in 20 minutes. It then immediately completed a backup of my MacBook Pro which had been waiting for the archive.

It seems to be fine now, but I'll send you a report in case that info is of any value.

I've been having one other strange recurring event. Whenever the iMac does a backup, at the time of the backup it logs a Power Management Scheduled Wake even for 7:58 the next morning. I have no qrecall events scheduled for that time, and don't think I have any other events scheduled then either, though i guess there could be something I'm not aware of.

I'll send a report from the iMac in case thats of use.

Ralph

Ralph,

It does, indeed, sound like the capture process was stuck. I can't think of any normal activity that would take that long to finish, even over wireless on a really large archive. The diagnostic reports probably won't help too much, as I really need to see a sampling of the process at that time, but I'll look for anything suspicious.

As for the iMac, the diagnostic report will include all of your active actions, so I'll be able to tell you what's going on there. That is, just as soon as I get back to the office, which will be a few days from now.