James Bucanek wrote:
In the second sample, QRecall is actually hard at work. It would appear from the log that your archive has accumulated a lot of empty space. QRecall keeps track of the unused portions of your archive and reuses it. As the number of unused areas increases, so does QRecall's overhead. It must periodically stop and clear the unused space that has accumulated?QRecall deliberately erases all deleted data from an archive. This is what it was doing when you made the second capture. This maintenance can occur at any time during a capture and it would appear that the capture is "hung up."

This also exacerbates another problem. If you terminate (force quit) a QRecall process, then next action has to auto-repair the archive before it can proceed. Among the auto-repair tasks is to re-erase all of the empty space in the archive. In your logs, I'm seeing that QRecall it taking upwards of 20-30 minutes to accomplish this, which I suspect means there's a lot of empty space in your archive.

Thanks for these explanations. One of the reasons I've force-quit QR as often as I have it that it often seems unresponsive to repeated cancel requests without giving any indication that it's still doing anything. If it's possible to give some kind of indication that the app is still productively working that would be useful.

Another example: I started a Compact action (running on the iMac where the drive is mounted, after turn off file sharing, rather than over the network from the MBP). I then realized that I hadn't merged layers for a while and it would make more sense to that before the Compact rather than after it. So I immediately tried to stop the Compact, and got no obvious response -- QR appeared to be in a hung state. In light of your explanation (above) though, rather than force-quitting, though, I just let it sit and went away for a while. It took over an hour for the Compact action to clearly terminate. I then ran my Merge action and followed it with Compact again, which is what it's doing now. If it hadn't been for your explanation I would have given up and force-quit the Compact long before it quit on its own.

Both QR and the Monitor window on the MBP continued to show the backup action as running, in spite of several cancel requests, quitting and restarting QR several times, the fact that the archive was no longer accessible. I had to restart the computer to reset QR to its proper state. I'm reporting these detail for your information, and don't need a response right now. Tomorrow after the Compact is finished I'll try another backup and I'll let you know what happens.

BTW, through all these problems with my MBP backups, the daily backups of the iMac have run without any problems.

Ralph

James,

Here's a sample from another hang-up, after running for about 7 minutes. Like the earlier on, it is one of two instances of QR Helper belonging to ralph, the one using more CPU and memory resources. This was using slightly more than 100% of CPU (on a 4 processor Mac) when I ran this, and is now down to 3%.

Let me know if you need anything else.

Ralph

James,

Once it starts the process seems to keep running until I force-quit QR Helper in Activity Monitor, or restart the computer, so the attached sample if from a process that's been running for 5 hours.

Activity Monitor does not show a QR Helper process running as root. Two versions are running, both as ralph. The one I sampled is taking up 0.1-0.3% of the CPU, and using about 1gb of real memory and slightly more virtual memory, with 7 threads. The other one has 3 threads, uses no CPU, and much smaller amounts of memory.

Let me know if you'd like to me restart a fresh process and sample that.

Ralph

I've had this intermittent problem crop up a few times, where qrecall hangs up when it starts to capture a backup. The monitor window opens the archive and shows "Capturing .config" and just sits there. It doesn't respond to cancel requests, although the log shows that they have been received.I have to force quit qrecall and qrecall helper, and sometimes even restart the computer, to get it back on track. It usually works the second time, though.

Today it was a bit different. Instead of hanging immediately when capture started, it captured 14.6mb and then hung, "Capturing Adobe Reader." Force quitting all the qrecall components and even logging out didn't help. Qrecall would start again showing that capture still running. I finally restarted the computer and that let me restart qrecall and the backup normally, but then it hung again while "Capturing .config" and is still sitting there after almost an hour.

The computer is a 2011 MacBook Pro running 10.8.5, backing up over wifi to a drive attached to an iMac. The iMac backups are fine, and have never exhibited this problem.

I'm sending a report from the MBP.

Ralph

I definitely had a sleepimage and swap files in my MBP backup, in /private/var/vm. They didn't come back in the next backup after I deleted them, though, so I guess it's excluding them now. Unfortunately, I didn't think to look at the dates of the files I deleted, and they don't seem to be showing up in "deleted files." The archive for my iMac did not include any swap files.

Ralph

It occurred to me recently that there's no reason to back up my sleepimage or my swap files, so I opened my archive and deleted those from the archive. Then I tried to exclude them from future backups, but since they're invisible system files it doesn't seem possible to do that. They don't show up in the Action OpenFile window, even when I've turned on "Show invisible files in Finder."

That left me wondering what other system files, etc., it's reasonable to exclude from my backups?

Ralph

I've been having problems recently on my MBP with intermittent Finder freezes, which can sometimes be cleared by relaunching Finder and sometimes require a complete restart. One occurred this morning as I was mounting a backup volume that qrecall was waiting for, and interrupted the start of the backup. (The Activity window was showing "Opening archive" when the freeze occurred.) Qrecall continued to show the backup as running, and I eventually had to force quit Qrecall Helper in Activity Monitor to clear it. Now when I try to restart the backup, Qrecall tells me that the archive is being updated by another action. I get this message on the iMac to which the backup drive is physically mounted, as well as on the MBP, which is connected to the iMac over wifi.

What can I do to "close" the archive so that I can access it again?

Ralph

3 hours later, the problem is solved but I'm not sure what happened. Here are the details in case they're of use to you. I decided to see if I could unmount and remount the backup drive, and if that would have any effect. Actually, I thought that would tell me where the archive was being held open (MBP or iMac), because the open status would keep it from dismounting. So I first ejected the drive from the MBP. That worked without incident. I then tried to open the archive on the iMac, and got the same error message saying the archive was in use. So I tried to eject the drive from the iMac. That failed, with a message that the drive couldn't be ejected because it was still being shared. (At this point, it had already been ejected from the iMac.) I turned off File Sharing on the iMac, and found I could now open the archive on the iMac. I turned File Sharing back on and remounted the backup drive on the MBP, and voila, everything works and my backup is now in progress.

I thought that since I had posted the problem I should update the post and tell you what happened. I'm a happy camper now and don't need any further answer about why. This confirms my past experience of the robustness of Qrecall, in that whatever happens to apparently screw it up temporarily, it always recovers. That's a really important attribute of the app. Thanks again, James.

Ralph

I stopped the interminable backup and switched the drive to connect directly with the MBP. From there, the backup captured 101696 items, 1.69 GB (94% duplicate), and took 45 minutes. It did not seem to be reading the whole drive this time, so whatever was causing it earlier didn't transfer to the direct mount. The interminable backup did not create a new volume, but just became part of the volume that was already there.

In any case, everything seems back to normal now.

Ralph

I seem to be running an interminable backup. I'm backing up my macbook pro drive with about 500g on it. yesterday the backup captured 904 items, 403.7 MB (70% duplicate), and took 8 minutes. Very little changed since then. Today's backup has been running for 8 hours and has captured 52gb (99% duplicate), 68,000 items. Activity monitor is showing a steady stream of received data at between 2 and 4 mb/sec. Is qrecall reading the entire archive? If so, why, and is it normal? I don't think I've seen this before.

I'm running this backup over a wifi network, right now showing a transmit rate of 52. If qrecall does need to read the entire archive I'd probably be better off stopping this backup and rerunning it with the drive directly connected to the MBP. If I do that, will the archive keep the material already backed up, or will it purge the uncompleted backup?

Ralph

After posting my previous message I read your previous exchange with Dr D and saw your suggestion to turn off the file server. I cycled file sharing on the iMac where the backup drive is mounted, and now it's working again. You solved my problem without even reading about it. That's really great customer service. Thanks.

Ralph

I accidentally closed my macbook pro this morning while it was backing up over the network, and now my archive seems to be hung so I can't access it. I've done this in the past with no problem. The backup usually just resumes when I open the lid. This time, though, it didn't, and when I put the cursor over the monitor window I got a beachball. Finder hung at about the same time. I attempted to start qrecall to see if I could fix it there, but qrecall hung as it was starting. When I force-quit qrecall it just started again and hung again -- over and over. I finally restarted the computer, and tried to restart the backup. The monitor window showed "waiting for archive. I tried to open the archive in qrecall, but it shows that "the archive is being updated by some other action. It will open when the action is finished."

I'll send a report.

So it's apparently hung in some kind of open state. How do I close it so I can resume my backup?

Ralph

So the status is "captured today" means that some computer has been captured to that archive today, but not necessarily the computer from which I'm making the query? My wife's computer backs up at 1am and my laptop backs up when I open it in the morning, so if my computer didn't back up for some reason, the status I would see on iy would still be "captured today?" I can live with that, it's just not what I thought I was seeing.

Ralph

The status windows shows only the status of local volumes. I back up both my computer and my wife's to the same archive, which works fine. I regularly check the archive status on my computer, but do so much less frequently on my wife's. I missed a week of backups on her computer because I forgot to un-suspend backups to my alternate backup disk when I swapped it in recently, and didn't check status on that machine during that week. If the information is in the archive, it would be nice if the status window showed the status of all volumes contained in the archive rather than just local volumes. Not a big deal, but a convenience.

Ralph

My experience using whole disk encryption for archives has been positive. I maintain a pair of archive drives, backupA and backupB, both of which are encrypted using Mt. Lion's whole disk encryption. I keep one offsite and backup to the other, switching them every week or two. It works fine. When I first mounted each drive I had to enter the password, but it's now stored in my keychain so I don't even have to do that. I keep the backup drive mounted on one computer, and backup a second computer to the same archive over wifi. Once the drive is mounted and the password has been entered, both computers access it transparently.

When I first set up this system I thought about giving both archives the same name and having only one action to back up to both, since only one would ever be attached to the computer. I asked James about that and he advised against it, saying the OS would recognize the drives as different and refuse to back up to one of them, even if they had the same name. I then set up separate actions for each drive, keeping one of those actions "suspended" and switching that at the same time I switch drives. Works great.

Ralph

I just finished compacting an archive and noticed that the Items column in all the layers now says "Incomplete." The first time I ran this compaction the backup drive accidentally got disconnected, requiring repair as a result. The "Incomplete" may have been introduced by the repair, but I didn't notice it till the completion of the compaction.

Is this a problem of any kind, or should I just ignore it?

Ralph