I've now completed a repair and it seems to be acting normally again.

Thanks.

I have two backup drive, keeping one off-site and swapping them every week or two. I did a swap yesterday, mounted the drive on my MBP and backed it up, then switched it to my iMac and backed that up. Both backups were uneventful and successful. (I normally leave the drive attached to the iMac and backup the MBP over the network, but usually do the first backup and a verify attached to the MBP when I swap drives because the MBP has USB3 and is faster. I didn't do the verify this time.)

This morning, today's (network) backup of the MBP failed showing "size of negative hash map not power of 2" and " A network or disk error was encountered." Those errors also showed up when I tried to backup the iMac (directly mounted) and when I try to backup the MBP with the drive mounted there. I get the same errors when I attempt to do a verify.

Disk Utility and TechTool Pro both show the drive as being OK. I've started a repair and that seems to be proceeding normally, but I wanted to let you know about this behavior in any case. I'll send a report from the MBP; if you'd also like one from the iMac let me know.

I've been reading about Ransomware attacks, where the hacker encrypts everything accessible from the hacked volume and then demands payment to decrypt the data. From some of the articles it sound like what gets encrypted might include a mounted backup volume, though I haven't seen this spelled out in detail. I'm guessing that an attached but unmounted volume might be invisible to the attacker and thus survive. What do you think about adding qrecall options to mount an archive volume before beginning a capture action and unmount it after that action as a defense?

If you can do that with an attached disk, can it also work for a disk attached to a networked computer? Mounting and unmounting of an attached disk can be done with Disk Utility, but I don't see any way to do that with a disk attached to a networked computer. I'd like that option because I leave my archive attached to one computer but also back up a second one over the network.

Ralph

I did forget to check "ignore ownership" when I mounted the new drive, so that probably was the problem. It all seems to be working OK now, though. Thanks.

Thanks for the suggestions. I?m back up and running now, but had one questionable event in the process. I wiped drive 3 (the new drive) and copied and renamed the archive from drive 2, this time deleting the status.plist to deal with the identity issue. I then mounted drive 3 on the iMac and went to bed, allowing the scheduled overnight backup to run. When I got up this morning it had run successfully.

I then attempted to backup the MBP over wifi (my normal backup procedure). This failed three times in a row, reporting ?cannot open negative hash map.? I dismounted the drive from the iMac and rwmounted it directly on the MBP, and the backup ran successfully. I then moved the drive back to the iMac and ran another MBP backup over the network, which was also successful.

I?m running a surface scan now using TechTool Pro, which is reporting no problems. I?m sending you a report from the MBP FYI in case the ?negative hash map? reports are of value to you. At this point I?m back to normal and happy, so I don?t need any more of your time on this.

Thanks again for the superb support you provide. It?s one of the best features of this great app.

I've been backing up both a MBP and an iMac to an alternating set of archives, keeping one offsite. My primary backup drive was getting old, so I decided to replace it in October with a new one. The new drive failed in December so I sent it back, and received a replacement on Monday, 2/2. I then copied my remaining archive (call it #2) over to the new drive (call it #3) to continue with my alternating backups. Copying was done using USB3 on the MBP. The copy looked fine, so I backed up the MBP to archive #3. That backup completed successfully. I then moved the drive over to the iMac (where it is generally mounted) and when to bed.

At 2015-02-03 01:00 the iMac attempted a scheduled backup, which failed. In the morning I saw that it had failed but didn't look closely at the log and reran it manually (starting at 2015-02-03 09:18 in the log).That capture also failed, apparently disastrously, so at 10:32 I ran a repair. The repair found large amounts of invalid data, and after it was over, all layers in the archive containing data from the iMac show up as damaged. I'm sending you a report from the iMac.

The most obvious potential source of this problem is my replacement drive, which Disk Utility shows as good. I don't know how much diagnostics the manufacturer can do if I send it back, since I encrypt my backup drives and would rather not give them the key. Can you tell if this problem was caused by my replacement drive, or by something else, and not the drive, what should I do next. If it was caused by the drive, can you suggest anything I can pass on the the manufacturer?

Other possibly relevant info. The MBP backed up successfully to the replacement drive, and I assumed that took care of the potential identity problem I asked about in <http://forums.qrecall.com/posts/list/495.page>. The MBP is USB3 while the iMac is USB2, if that matters for any reason. Both machines have been routinely backed up to my other backup drive without issue, and the replacement should have been an exact copy of that drive.

Any light you can shed on this would be helpful.

Thanks, Ralph

OK, thanks. Normally both archives aren't in the house at the same time but they will be when I'm copying onto the new drive, so I"ll do something then. I thought I remembered something about the archives have internal identifiers, so I'm glad I thought to check.

Ralph

I backup to two distinctly named archives on two separate drives, swapping between them and always keeping one offsite. One of those drives just failed and was replaced with a new drive. Can I simply copy my remaining archive over to the new drive and rename it as my second archive, or is there internal identification information within the archive that would create problems doing this?

Ralph

Thanks for the feedback, James. I still have a couple of questions, primarily to fill in my understanding of how Qrecall works.

After the repair I ended up with two layers marked "damaged" and one marked "incomplete," if I'm remembering it correctly. Merging each with the next later layer I ended up with two "damaged" layers, which seemed to absorb the following good layers into the damaged layers rather than the reverse. (I'm not sure now if one of the was the originally "incomplete" layer, or if they were both damaged layers.) Rereading your post, it looks like that was because the subsequent layer didn't yet contain all the damaged files, and that there's no real downside to leaving them that way. Is that conclusion correct?

After my uncompleted Compact action, the unused space shown in the Status window jumped from less than 10mb to 106gb. If I understand Compact correctly, that was space that had been occupied by previously deleted files which the compaction process had erased, leaving scattered free space throughout the archive, and if the process had completed properly, those spaces would have been closed up by shifting everything to fill them in. Currently, though, qrecall simply uses that as available space within the archive. If this is correct, is there anything to be gained by running Compact again to close up those spaces, or is it just as efficient to simply let Qrecall fill them as over time?

Ralph

I ran a compact action on a 1tb archive yesterday (2014-12-01 20:40:22), which freed 106gb of space but quit before actually moving files to compact the archive. I then attempted a backup (2014-12-02 03:00:00) which didn?t run, with some connection problems between the drive and the computer which apparently corrupted the archive. My memory isn?t precise here, but I think the drive somehow unmounted itself from the computer. I repaired the archive (2014-12-02 08:03:54) and that repair ended with the drive again unmounted from the computer and the log reporting ?Data Problems Found?

Not knowing what else to do, I repaired the archive again (2014-12-02 13:33:43), and this time the repair seemed to work, though the log again listed long lists of ?Data Problems Found? (2014-12-02 15:11:27). I followed that by another backup that completed successfully.

My primary question now concerns the integrity of the archive. Does the fact that the second repair completed successfully imply that my archive is intact, or does the long list of ?Data Problems Found? imply that the repair process may have discarded parts of the archive? I can live with it either way, since I have another good archive of the same computers, but I want to understand what happened.

This is the same archive that I?ve had intermittent problems with in the past when backing up over a network, but in this case the backup drive was directly connected to the computer during all these operations. I don?t know what caused the instability with the connection between them. This drive is less than two months old, so I don?t have a solid history with it yet, but it seems to be working OK. This experience does make me feel good about having two different back sets that I swap between, though.

I?m submitting a report on these activities. The date/times in parens above are from correspond to the qrecall log entries for those particular actions. Things seem to be back to normal now and I have no urgent need for a response, so let this be a low priority for you.

Ralph

Thanks, James. I always find it interesting to learn more about Qrecall's innards. I verified the archive to see if it was OK or not, and it wasn't. So I repaired and now it's fine.

I backup my MacBook Pro over my LAN to a backup drive mounted on another computer. I occasionally have backup failures due to transient network problems, including operator error when I close the lid during a backup. These failures show up in the Archive Status box with a yellow circle and a "minor problem" message. Rerunning the backup is usually successful and clears the status indication.

On Wednesday I accidentally closed the lid during a backup and got a status window message that the archive required repair. Since that's a long process with a terabyte archive, I decided to rerun the backup first. It completed successfully and has been working since. The Archive Status window, though, refuses to reset and continues to show an "archive needs repair" message and a red circle, as does the status window on my other computer. I even told the Status window to forget the archive, thinking that would clear it, and the repair needed message still came back. Is the Status window indicating an underlying problem with the archive that needs repair, even though it seems to be working OK, or is this just a problem with the status report? I've sent a diagnostic report from Qrecall.

Since updating to Yosemite, I'm seeing a couple of other anomalies in the log. The first is repeated "Cannot connect with scheduler" messages, followed by reinstallation of com.qrecall.monitor.plist, and the second is an "Unable to determine path to VM swap files; assuming /var/vm" message within each backup log. (I don't see either of the log entries in my other computer.)

And as long as I'm writing, I'm curious about what's reflected in the "Ignored some changes" entries I see frequently?

Remember that the merge, compact, and verify actions apply to the archive as a whole and not to the individual computers, so if you're backing up multiple computers to the same archive you can perform those actions from a hard-wired machine even if you're backing up laptops over wifi.

If you back up laptops to separate archives it may still be possible to merge, verify, etc. those archive from a wired computer on the network. I'm not sure about that, though, since I don't do it that way.

OK, thanks. I tried the control key inside the Qrecall list and didn't think about the other possibilities. I'll exclude the plist.

James Bucanek wrote:
I don?t have much in the way of an explanation for why your layers disappeared on you, other than to suggest that you make sure you?re viewing the current owner and volume. If you have the Hide Unrelated Layers view option selected, and you inadvertently select an older owner or volume in the archive, QRecall will ?conveniently? hide all of the layers since that volume was last captured.

That probably was the cause of my disappearing archive. The archive I was left with was from my previous MBP, which I retired in December. I had never noticed that hide command but must have triggered it inadvertently. I'm feeling a little stupid right now for panicking when the archive seemed to disappear, instead of investigating it a bit more carefully.

Reviewing the logs for your iMac, you?re consistently getting capture failures, but they?re rather minor. In fact, going back almost a month, the only consist problem you?ve encountered is capturing the tmSync.plist file in your iPhoto database folder. It would appear that some process is constantly replacing this file. When QRecall goes to capture it, it reads the directory, but by the time it gets around to reading the file, it?s been deleted and replaced. QRecall logs this and continues. It doesn?t, however, have any impact on the integrity of the archive.

(If you find this annoying, I might suggest that you add the tmSync.plist file to either the capture's ?exclude? or ?ignore changes? list.)

I haven't noticed problems with the iMac backups, except for the occasional problem notice in the status window when nothing showed up in the log. The iPhoto Library is a package rather than a folder, and there doesn't seem to be any way to drill into it to exclude the plist file, so I guess I'll just leave that as is.

The MacBook Pro is another story. As you mentioned, it consistently fails (about once or twice a week) with I/O errors while reading or writing to the archive over the network. I/O errors are a deal breaker and terminate the capture, leaving the archive in a state that requires repair. Typically, the auto-repair is sufficient to get it back in service. But you also occasionally experience permission errors. I can?t tell if this is a file server issue, a virus scanner, or genuine permission problem. Permission or other access failures can leave the archive is a state that requires a full-blown repair or reindex to get things back into shape.

Which brings us to the repair actions. On the few occasions that you?ve perform a repair, the repair logged nothing out of the ordinary. This implies that the I/O and permission errors you?re encountering on the MBP are due to transient network transmission errors and are not a result of any damaged data in the archive or issues with the physical media.

If you?re ever concerned that your archive might have been compromised, run the Verify command/action.

The problems MBP have been minor but manageable, and usually resolvable by simply rerunning the backup. The failures in the last couple of weeks have required reindexing, which is new. I've been repairing rather than just reindexing because of past experiences when that was required, but it sounds like reindexing would have taken care of it in these cases. I generally Merge layers and Verify the archive every couple of weeks, when I swap in my off-site backup drive.

One thing I will mention: I see that a Repair action was run from the MPB to an archive that (I assume) was accessed over the network. This is asking for trouble.

Both my MBP and that backup drive have USB 3 while the iMac is USB 2, so I sometimes physically mount that drive to the MBP for long operations like a repair, to take advantage of the higher speed.

Thanks again for the prompt and useful support, which you always provide.

Ralph