My backup drive is normally attached to my wife's iMac, and my MBP gets backed up to that drive over the network. As you know, I've had occasional problems with scheduled MBP backups not completing properly, then completing fine when I rerun the backup. I attribute this to network flakiness and it's not a big problem.

This week, though, I'm having more consistent problems. I've had several backup failures on both machines, have repaired the archive three time, and am being told that it needs to be reindexed again. The problems appear primarily associated with the MBP, but after two apparently successful iMac backups the Status window showed backup problems eve though the log showed nothing. At one point yesterday all my layers after Nov 2013 disappeared, but repairing the archive brought them back.

I'm sending reports from both computers. If you need a more detailed description of my experience let me know.

Ralph

I back up my MacBook Pro over wifi to a backup disk connected to my wife's iMac, using a backup action set for 3am and "hold while no archive." The MBP lid is often closed at night, so the backup takes place when I open the lid in the morning. I've been doing this for years, through three different laptops, and it usually works flawlessly. Since getting new retina MBP in December, though, I occasionally find that the MBP has attempted a backup sometime in the early morning (not at the 3am scheduled time) while the lid was closed, and that backup has failed. I can then successfully rerun the backup manually. I'm guessing that this is somehow related to Apple's new "Power Nap" feature, even though that option is not checked in my Energy preferences. This isn't a big deal, but I'm passing this on FYI. I'll submit a report that shows it in backups on 4/7 and this morning.

I have Sophos anti-virus running and watching my email, and it occasionally quarantine's Windows viruses and asks me to remove them. If a backup runs before I've done that, a log entry will show up that the file could not be captured. See this morning's 9:43 backup, for example. Again, not a big issues, but just FYI.

Ralph

When I look at the number of times that my name is cropping up on the "Problems . . ." Forum, I feel like some kind of complainer, but I just seem to be having a lot of minor, intermittent problems recently. These problems, including this one and the "hash file" issues I've written about previously, intermittently prevent backups from completing. They don't seem to damage the archive, though, because if I just restart the backup it will run to completion without an issue. My wifi network probably is involved, since it doesn't happen to backups of the iMac on which the backup disk is mounted, or to occasional backups to my MBP with the backup drive directly mounted. It does seem to be happening more often, though, and I've been running the network the same way since I started using qrecall in 2007.

This latest problem, where qrecall stops processing a folder and reports "cannot read envelope content length" has occurred eight times since Nov 26. Sometimes it affects a single file or folder, other times qrecall will run through and balk at a whole series of folders. During the same period, I've had six failures due to "negative hash file" problems. After either of these failures, a rerun of the backup will generally be successful. I don't think I saw these problems before Mavericks came out, and most of them occurred since I got a new retina MacBook Pro, so both of those factors may be involved.

Qrecall's lack of responsiveness to cancel requests is also a minor annoyance. Often time when the backup hangs, qrecall refuses to terminate the process and I need to terminate qrecall helper manually in Activity Monitor. (I know it sometimes takes a while, but I'm talking about 15 minutes or more here.) Sometimes even this won't work, and the only way to regain control of Qrecall seems to be by rebooting.

I'll send a report.

Ralph

Thanks for the suggestions, James. I'll experiment some more. I think that one or two of the failures you noted resulted from my disconnect a backup in process by inadvertently shutting the laptop lid. Sometime a backup in process seems to survive this, and other times it doesn't.

My major concern is the the "failure to close negative hash map" errors, which do seem to be network related. While the log then reports "The archive may need to be repaired," when I've tried to rerun the backup without doing a repair it does seem to work.

Steve asked about my network details. My wifi is served by an Apple Airport Extreme (802.11n) which is several years old. The archive is mounted on an iMac connected to the AE via ethernet. Both computers are running Mavericks. I've read that Apple switched the network protocol in Mavericks from AFP to SMB2. I know nothing about network protocols so don't know if this switch could be involved, but I think my hash map problem started occurring after my upgrade to Mavericks.

I'm due to retrieve my off-site backup tomorrow and swap it out for this one, so that will let me see if the drive mechanism may be involved.

Thanks for the continuing support; I appreciate it.

Ralph

To update my previous post, yesterday I again mounted the backup drive directly to the rMBP and ran a backup, without first repairing the archive. The backup ran fine, and took 6.5 hours to back up 203gb with 99% duplication, out of 450gb on the computer. I guess it was backing up so much because the computer is new, even though the same data had been previously backed up from a different computer.

So it does look like the hash map problems I'm having are network related, though the network setup is the same as I've been using for years. Do you have any suggestions of ways I can troubleshoot this further? I'd rather not have to mount the backup drive to the laptop for backups.

I'm continuing to have problems backing up my MBP over wifi. I got a new retina MBP this week and migrated to in, then connected my backup disk directly to consolidate a bunch of volumes that had accumulated over time from updates, etc., and do my first backup of the new machine. That went fine. Since then two backups over wifi have failed due to hash map problems. So the problem appears to be with my network, but I've been using qrecall this way since it was in beta and the problem is fairly recent. Do you have any suggestions of things I might do to tweak the network? I'd hate to have to go to manually mounting the backup disk in order to use it.

Ralph

I don't routinely run any virus scanning software, but do use Sophos Anti-Virus to scan incoming email for malware, and that's what picked up the viruses. I doubt that that was involve in the hash map problem. I'll send diagnostic reports from both my MBP, where the issue with qrecall not being able to process PC virus files occurred, and also from from my wife's iMac, covering a couple other bad backups this week.

The 1am scheduled iMac backup on 11/28 failed due to a permissions problem. Backups on the MBP are scheduled for 3am, but normally run when I open the lid in the morning. That morning at 7am the next MBP backup failed, logged as an incorrect index.

I wasn't paying much attention to the computer on Thanksgiving and didn't notice these failures, which led to another 1am iMac backup failure on 11/29, logged as "archive file out of sequence," followed by a 7:30am failure on the MBP logged as "header file length invalid."

I finally noticed the problem on Friday (11/29) afternoon and repaired the archive, after which both computers were successfully backed up. I'm sending you reports from both.

Question: The logs on both computers told me that the index is incorrect and needs to be recreated. I just repaired the archive, since I've found that reindexing always seems to require a repair anyway, so the reindex step is just a waste of time. Does it ever make sense to do a reindex rather than just doing a repair?

Ralph

These are minor issues that I don't see as problems, but things you might be interested in.

I've had a couple of backups fail recently due to hash map problems -- failure to open the hash map in one case and failure to close it in another. In both cases I reran the backup and it ran without problems.

Several backups recently have reported an inability to open an email attachment for backup. The unopenable files were ones that the Sophos Quarantine Manager reported as malware, so this was a good thing. It's not something I've seen until recently, though.

Ralph

James,

Here's a bit more information. I alternate two different backup drives, keeping one offsite and swapping between them every week or two. The larger captures generally represent the first backup after a drive was swapped in, with the 18GB capture on 10/25 being the first time that my 1st backup drive had seen Mavericks. I next swapped drives on 11/08, which is when the first excessive capture (59GB) occurred, and that's when the captures started getting bigger. On 11/13 I stopped the backup after more than 10 hours, when it had captured 64.4GB with 100% duplicate. The next day I restarted it and let it run, which was the one that took 16.5 hours to capture 139GB with 2GB written. The next backup was normal, as was today's.

And thanks again for the support you provide.

Ralph

I back up my MacBook Pro over wifi to a backup drive connected to an iMac. My computer usage is fairly light and my daily backups usually take under an hour, or maybe 2 at the most. I?ve had a couple of strange backups this week, with qrecall capturing what seemed like an extreme amount of data almost certainly from files that had not changed. Today?s backup seems back to normal, so I?m just reporting this (and sending you a report from the computer ) in case the data is of value to you. I?ll let you know if the behavior recurs.

My daily backup on 11/12 failed due to a network problem. I reran that backup and it completed successfully, taking about 2 hours. The next day, 11/13, the capture seemed to be going on forever. I cancelled the backup after 10 hours, when it had captured 64.4gb and written 620mb, with 99.52% duplicate. When I reran the backup the next day it took over 16 hours, capturing 138.7gb and writing 1.9 gb, with 99% duplicate. Today?s backup seemed normal, with 2gb captured and 108mb written, with 95% duplicate. (The drive being backed up contains 522gb.)

That's reassuring. Thanks for the clear explanation.

James Bucanek wrote:

After formatting the new drive, and before restoring, make sure the "Ignore ownership and permission" option was not checked.

Please confirm that this admonition applies only to Restore and not to Capture. I back up two computers from two different user accounts to the same archive, mounted on one of those computers, and find that I need "ignore ownership . . ." to make that work.

I had a problem again today with Qrecall hanging up during a capture. It's all cleared up and not currently a problem, but I'm passing on this history FYI in case it's of value to you.

This time it happened on my wife's iMac, during a backup that startedat 4:15am, after the completion of Merge and Verify actions. When I looked at the computer some time after 10am it showed 27 items captured, 273mb, 80% duplicate, after running for more than 6 hours. These backups usually take much less than an hour. I left it and came back again at 12:30, to find the status unchanged. At that point I sent several cancel requests, to which qrecall did not respond. Earlier in this thread you said that qrecall could take some time to quit normally, so I left it alone to see what would happen.

When I came back to the computer at 10pm, the status was still the same. I terminated the process by quitting (not force-quitting) qrecall helper from Activity Monitor. It terminated immediately, and logged the completion of the interrupted backup. I then immediately restarted the backup, which ran normally and completed in 20 minutes. It then immediately completed a backup of my MacBook Pro which had been waiting for the archive.

It seems to be fine now, but I'll send you a report in case that info is of any value.

I've been having one other strange recurring event. Whenever the iMac does a backup, at the time of the backup it logs a Power Management Scheduled Wake even for 7:58 the next morning. I have no qrecall events scheduled for that time, and don't think I have any other events scheduled then either, though i guess there could be something I'm not aware of.

I'll send a report from the iMac in case thats of use.

Ralph

OK, Thanks. I'll set up a nightly compact action and watch it for a while.

Ralph

Compacting my archive took 22 hours and recovered 64gb from an 875gb archive. Both computers have since backed up with no problems.

James Bucanek wrote:I highly recommend that you create a compact action and schedule it to run occasionally (say, once a month) for optimal performance.

I haven't been compacting because I was under the impression that qrecall would use empty space in the archive as it became available, but I guess I should change that behavior. Is there any simple way tell how much empty space there is in an archive?

Ralph