I have two drives containing essentially the same archives, one of which is normally stored offsite. I followed your direction to have both of them mounted when I first ran v2.0. I then started a backup into one of the drives, and looking at things like the archive settings on the other, and decided I could probably turn on Error Correction on the second archive while I was still backing up on the first. Initially, that seemed to work, but at about 56gb in to a 1tb archive, qrecall crashed and the error correction froze. The backup on the first drive is still running, so I'm assuming that it will complete satisfactorily. (Activity Monitor shows Qrecall as "not responding" and shows three instances of Qrecall Helper running, two active and one inactive. I don't want to force quit quit while the backup is running, but once it finishes I'll restart qrecall and send you a report. I'm running OSX 10.11.1 on a 2014 rMBP.

Question: Once I get this computer backed up to both archives I have a second computer to back up to both as well. Should that computer, as well, have both drives mounted when I run v2 for the first time?

Ralph Strauch wrote:(Activity Monitor shows Qrecall as "not responding" and shows three instances of Qrecall Helper running, two active and one inactive. I don't want to force quit quit while the backup is running, but once it finishes I'll restart qrecall and send you a report.

It sounds like both actions might still be running. A capture runs as two instances of QRecallHelper (one to perform the capture and a second to gather user-centric metadata; the latter is usually not very busy), and the error correction task runs a single instance. So my advice is to let them run. Eventually the QRecallHelper processes will run their course and you can check the log for success, failure, or incompleteness.

Why the QRecall app itself it hung is a mystery. If you have the Activity monitor open and QRecall is still "not responding" take a sample of it and email it to me, if you have the time.

Question: Once I get this computer backed up to both archives I have a second computer to back up to both as well. Should that computer, as well, have both drives mounted when I run v2 for the first time?

It's easiest if your archive are all reachable when QRecall 2.0 is run for the first time, but it is certainly not a requirement. You can easily make these updates yourself at anytime. Here's what the "upgrade" process consists of, in a nutshell:

Replace the legacy alias structure in the action document with a modern URL bookmark. You can do this by opening each action, reselecting the archive, and saving it again.

Move any excluded items from the archive's capture action to the archive's settings. To do this yourself, simply open the archive and edit its settings to set the you items want excluded.

Move any ignored items from the archive's capture action and turn them into capture preferences for that item. To do that yourself, choose the items you want ignored in the Finder and use the new QRecall Capture Preferences service.

Finally, aliases can be finicky. Even if you let QRecall perform the 2.0 upgrade, you might have better success with your rotating set of archives if you manually open their actions, reselect the archive, and save them again. That way, you'll know QRecall is storing the correct bookmark for each archive.

Here's the rest of the story. The backup continued to run and I watched its progress in Qrecall Monitor. When it got about 1/8" from the end of the status bar it stopped progressing. I checked Activity Monitor and it showed Qrecall Monitor also "not responding. so I force-quit both Qrecall and Monitor. I tried to restart Qrecall and couldn't, so I shut down and restarted the computer. As I was doing this I found most of the other apps I was running had also frozen and I ended up doing a hard power down reset. I don't know whether Qrecall was the cause of all of this, or it was just coincidental.

I'm back up now and it looks like my backup was completed, according to the log. I just sent a Report. At some point I'll try to verify the backup and see what I get, but I may go ahead first with a backup to my second archive, since the backup operation itself seems OK. I won't do anything more to install Error Correction until I hear from you.

I just tried to backup to my second archive, the one where I had the problem installing Error Correction. Qrecall now tells me that that archive needs reindexing, so I've just started a repair. Qrecall Monitor is showing Idle, while Qrecall is showing the repair in progress. I don't remember if that's standard, or not.

Ralph Strauch wrote:As I was doing this I found most of the other apps I was running had also frozen and I ended up doing a hard power down reset. I don't know whether Qrecall was the cause of all of this, or it was just coincidental.

I'm not aware of anyway that QRecall could cause your other apps to hang, so I'm going to go with "coincidence."

Ralph Strauch wrote:I just tried to backup to my second archive, the one where I had the problem installing Error Correction. Qrecall now tells me that that archive needs reindexing, so I've just started a repair.

According to the log you sent me, the capture action completed but it looks like the error correction change was still running when your restarted your computer. A reindex should fix it.

Qrecall Monitor is showing Idle, while Qrecall is showing the repair in progress. I don't remember if that's standard, or not.

That's normal. Actions that modify an archive run in the archive browser window, if that's where you started it.

It's been a busy evening. I started to write this response before your last reply, which came in the middle of the chaos described below.

The repair I started on the drive with the Error Correction problem has been running for almost six hours and I just force-quit it because it's generated a log of more than 45GB so far and if I don't turn it off before I got to bed it will fill my drive before I get up. It seemed to go through the first part of the repair cycle normally, then when it switched to "reindexing . . ." it started generating lines in the log that shows up in the app like this

Action 2015-10-30 21:40:36 Warning ignoring problem reading checksum; assume data is valid
Action 2015-10-30 21:40:36 problem reading checksums
Action 2015-10-30 21:40:36 Position: 483863340
Action 2015-10-30 21:40:36 Length: 4

thousands of them, one after another. The basic interval between one Position and the next was 4, the position number itself changed from one block to another from time to time. When TechTool Pro gave em a low disk space warning I checked and saw my disk space disappearing at a notable rate. When I shut it down the Qrecall Log in ~/Library was over 45gb. Qrecall didn't respond to the stop button, so I had to force quit it.

The log in the app now shows only a segment of the repetitive lines; the earlier stuff before that didn't survive the shut down. I started to send a report, but stopped it when it was still "compressing" after half an hour. I was in the middle of watching something on TV, running back and forth during commercials. When I came back at the end of the show, Activity Monitor showed qrecall still taking a lot of cpu, then shortly thereafter I got a system message that I was out of system memory because of Qrecall and needed to force quit applications, most of which were showing "paused" (including Safari, where I had most of this post written. Luckily I was able to resume it.)


So my current situation is that I'm sitting here with my backup drive unplugged and Qrecall not running. If I mount the drive it makes clicking noises even though nothing on the computer seems to be activating it. If I start Qrecall (with the drive mounted or not) and do nothing else with it, Activity Monitor shows it as "not responding" and gobbling up increasing amounts of memory (over 20gb not doing anything.

This computer now shows the 45gb Qrecall log generated while the archive repair was reindexing, along with a couple of smaller logs generated after that. Let me know if you would like any of that material. or other information I can provide. I'm about to head off to bed now, and will pick this up in the morning. I'm thinking now that my next step should be to reinstall qrecall and see if that lets it start without immediately gobbling up memory. Is that a good starting point, or not?

If I can get qrecall running properly again, then I can worry about whether this archive can be save, or will just have to be junked. I'm very glad right now that I had a pair of backup drives, and ran a backup using v2.0 on the other drive before these problems occurred. It's mounted on my other computer right now doing a Verify, which I hope will come out OK.

I just started qrecall to see what would happen. Watching it on Activity Monitor it just began to gobble up memory, even though I wasn't asking it to do anything. Sample from Activity Monitor attached. By the time I force-quit it, it was using 32gb of memory, I only have 16.

Ralph,

Don't worry. You're the unfortunate victim of the unfinished action that was adding error correction codes to the archive. Nothing you've done has made any changes, or endangered, the data in your primary data file.

Here's what happened. The action to add error correction codes to your archive crashed (or was terminated during the reboot). The correction code files are now incomplete. When you tried to reindex the archive, QRecall—in the interest of full disclosure—started logging every missing checksum and error correction block it found, or didn't find in this case.

That caused your log to explode. When you opened the QRecall app, you probably had the log window open. QRecall started reading the gigabytes worth of new log messages into memory. This began to eat up all of you memory and drag your system into the mud.

Here's how to get out of this mess:

First, get rid of the bloated log file. Open your ~/Library/Logs/QRecall folder and trash anything that looks too big. That's probably just going to be your primary QRecall.log file, unless you rotate log files daily.

Select your archive in the Finder. Control/Right-click on the archive and choose Show Package Contents. Inside the archive you're going to find a repository.data file. You'll also see a bunch of "companion" files with funny names like this:

repository_8k.checksum32
repository_p8w8k16m2.0.anvin_reed_sol
repository_p8w8k16m2.0_8k.checksum32
repository_p8w8k16m2.1.anvin_reed_sol
repository_p8w8k16m2.1_8k.checksum32

Trash all of these "companion" files. (Do not trash your repositoy.data file. That's where all of your data lives!)

Launch QRecall. Choose File > Repair Archive…, choose your archive, select Reindex, and Bob's your uncle.

I threw away the files you suggested and repaired the archive, and everything seems fine. I'm doing another backup now. The repair log listed about 3 dozen instances of "invalid data," mostly just a few hundred bytes but one chunk of half a megabyte. Should I be concerned about these at all.

Qrecall continues to be a great app, and one of it's finest features is the quality and responsiveness of the support that comes with it. Thanks again, James, for all you provide.

Ralph

Ralph Strauch wrote:I threw away the files you suggested and repaired the archive, and everything seems fine. I'm doing another backup now. The repair log listed about 3 dozen instances of "invalid data," mostly just a few hundred bytes but one chunk of half a megabyte. Should I be concerned about these at all.

What you really want to look for are indications of incomplete folders or files that were a result of those stretches of "invalid data". These will also be in the log. If you don't see any messages like that, then the stretches of invalid data were probably detritus from an incomplete merge or deleted items (as a result of an earlier merge) that hadn't yet been erased.

Qrecall continues to be a great app, and one of it's finest features is the quality and responsiveness of the support that comes with it. Thanks again, James, for all you provide.

You're very welcome.

Is it safe to make another try at adding error correction now, or should I wait for the next release?

Ralph Strauch wrote:Is it safe to make another try at adding error correction now, or should I wait for the next release?

As far as I can tell, your issues with adding error correction was in the QRecall app (the monitor), not the action (the doer). It appears that the action continued to run (until the computer was restarted), as I can find no crash logs or other evidence that it died abnormally.

I'd suggest you give it another run. Just remember that it has to read the entirety of the archive data, so it will take about as long as a verify.