Ralph Strauch wrote:I threw away the files you suggested and repaired the archive, and everything seems fine. I'm doing another backup now. The repair log listed about 3 dozen instances of "invalid data," mostly just a few hundred bytes but one chunk of half a megabyte. Should I be concerned about these at all.

What you really want to look for are indications of incomplete folders or files that were a result of those stretches of "invalid data". These will also be in the log. If you don't see any messages like that, then the stretches of invalid data were probably detritus from an incomplete merge or deleted items (as a result of an earlier merge) that hadn't yet been erased.

Qrecall continues to be a great app, and one of it's finest features is the quality and responsiveness of the support that comes with it. Thanks again, James, for all you provide.

You're very welcome.

Fixed.

And I'm so glad you asked.

Ooops,

I wrote too soon. I just tested this out, to make sure it worked, and apparently it doesn't.

If you capture a folder, and that folder is set to ignore changes, the changes may, or may not, be ignored (it's a bug and it depends on where the folder is in the hierarchy).

I will definitely make sure this is working before the next version is released.

Alex wrote:- First, despite rebooting, copying QRecall.app around, etc. I can't get LaunchServices to see the new "QRecall Capture Preferences" service. I do have "Capture to QRecall Archive", etc.

You might have to explicitly enable these new services in Finder > Services > Services Preferences…. If they're not listed there, you might have a "stuck" copy of the old services in ~/Library/Services/. Delete the QRecallService.service package from this directory and launch the QRecall app again. QRecall will reinstall the new service and notify OS X that it is available. Afterwards, it should appear in your Services Preferences.

- While I understand the value of exclusion filesystem attributes for items that may move, I'm actually much more concerned about never capturing items at particular paths. These are folders that may be deleted and recreated, and thus attributes will be lost. The good news here is that per-archive exclusions may work, except...

- The loss of exclusions per capture mean that one cannot set independent schedules for an item that is a subitem of a larger capture. For example, I backup my boot drive frequently, but exclude a subdirectory of VM images. However, previously the VM images could be captured on a separate event by a capture with a different exclusion set.

This is exactly the "mistake" that the new exclusion arrangement is designed to prevent you from making. Here's the problem with the way it works in QRecall 1.2:

You capture your home folder every hour, but exclude you VM folder.

Every evening, you capture your home folder but do not exclude your VM folder.

The problem with this arrangement is that your VM folder alternately appears, and then disappears, from your archive. After the first capture your VM folder would not be captured. After the second, it would. If your repeat the first capture again it's gone again. If your hard drive crashes and you restore from the latest capture, your VM folder would not be restored—because, as far as QRecall is concerned, it didn't exist when the third layer was captured.

The second problem this creates is that your VM folder get repeated captured in its entirely, rather than intelligently capturing just the changes. When the second capture runs, the VM folder doesn't exist in the archive history, so all of the files are captured as if they were brand new. After the third capture (that excludes the VM folder), the VM folder is erased in that layer. The next capture of the VM folder starts over from scratch because7mdash;again, from QRecall's perspective—all of the files are new.

What you really want to do is use the Ignore Changes feature. Set your VM folder to ignore changes for 20 hours or so. During the day, your startup volume or home folder can be quickly captured without trying to recapture your VM folder.

You can then create a second capture action to capture just your VM folder at the end of the day, or trigger that to run when your VM application quits.

Tip: I'd also suggest setting the "Deep Scan" option for the VM folder capture action, as VM packages (particularly Parallels) are notorious for tricking the filesystem change history into not seeing all of the changes.

Note that the "Ignore changes" option for capture preferences is not a good replacement, as it forces me to only ignore on a schedule whose timeline is opaque, I can't force a capture on an event (say, when VMWare quits).

Good point.

That's exactly why there's an exception to the rule: an ignore preference set on a folder is ignored when that folder is explicitly a target of a capture action. In other words, the per-item ignore preference only applies to items contained in an item being captured, never to the items listed in the capture action.

Finally a couple of UI issues:

- The exclusions scroll view in Archive settings is small, and cannot be resized. For exclusions inside ~/Library or long lists it's a frustrating interface to verify that all the items you want are present. This was not an issue with the prior interface in the capture action.

I've added this to the to-do list for QRecall 2.1, which will be a major UI facelift.

- If one does choose to use capture preferences as extended attributes, there's no simple UI to actually see what's excluded. Again this is necessary to verify you haven't missed something that should be excluded.

You can do this with the command-line tool. For example, this command will list all of the items contained in your home folder that have capture preferences set:

qrecall captureprefs list --recursive --skipnormal ~

Charles Stembridge wrote:How "beta" is 2.0? I'm hesitant to trust backups to a beta version.

It's actually very close to a release version. There are still some odd issues getting the scheduler run when installed, a few minor UI problems, and missing documentation. But there are currently no known issues that impact data integrity or reliability.

Charles Stembridge wrote:Any idea as to what I did wrong there?

I honestly don't know. Mail is really scattered all over the place now. Mail accounts are defined in the system's Internet Accounts database, preferences are in several locations, etc.

I helped a friend replace the SSD drive in your iMac and we could not get Mail running again after upgrading to El Capitan. I eventually relocated the Mail folder to the desktop, deleted and recreated all of her mail accounts (that restored her IMAP mailboxes), and then imported the local mailboxes back into Mail from the copies on the desktop. That seemed to work pretty well.

If you have captured copies of your mailboxes, you could try recalling your ~/Library/Mail folder to your desktop and try recovering mailboxes manually by reimporting.

Charles,

You didn't mention which version of QRecall you are using. If it's 1.2, then restoring a bootable Yosemite volume is dicey. Starting around OS X 10.10.3, Apple started adding a lot of new security requirements that a volume restored with QRecall 1.2 might not satisfy, and it's completely incapable of restoring an El Capitan system.

You did the right thing restoring the volume—boot from an external volume and restore your boot volume. Just make sure that your external volume has the same major version of OS X that you're restoring. It's often the case that an older version of OS X (10.9) will be incapable of restoring a newer version of OS X (10.10), because of new features Apple has added to the filesystem.

OK, so you've restored your boot volume and it doesn't boot. Don't worry. Starting with OS X 10.9, the best remedy is an over-install. After recalling your boot volume, boot into your OS X recovery partition and simply reinstall OS X over what you recalled. OS X is now very good about surgically reinstalling the OS. It won't overwrite any of your accounts, documents, applications, or preferences. When the install is finished, you'll have the latest OS and everything else should be intact.

Now, about capturing Mail. The Mail app uses a database. This is really difficult (read "impossible") to cleanly capture while you have Mail open. The best time to capture your mail is when the Mail app is closed.

If getting a clean capture of your mail is really important, and you're using QRecall 2.0, consider scheduling a capture of your mail folder and preferences "when Mail closes", using the new application-based event schedule. You can also have QRecall skip your hourly capture if the Mail app is open, or just hold until the Mail app closes. But if you're like me and leave your mail app open all day, that won't be of much help.

Ralph,

Don't worry. You're the unfortunate victim of the unfinished action that was adding error correction codes to the archive. Nothing you've done has made any changes, or endangered, the data in your primary data file.

Here's what happened. The action to add error correction codes to your archive crashed (or was terminated during the reboot). The correction code files are now incomplete. When you tried to reindex the archive, QRecall—in the interest of full disclosure—started logging every missing checksum and error correction block it found, or didn't find in this case.

That caused your log to explode. When you opened the QRecall app, you probably had the log window open. QRecall started reading the gigabytes worth of new log messages into memory. This began to eat up all of you memory and drag your system into the mud.

Here's how to get out of this mess:

First, get rid of the bloated log file. Open your ~/Library/Logs/QRecall folder and trash anything that looks too big. That's probably just going to be your primary QRecall.log file, unless you rotate log files daily.

Select your archive in the Finder. Control/Right-click on the archive and choose Show Package Contents. Inside the archive you're going to find a repository.data file. You'll also see a bunch of "companion" files with funny names like this:

repository_8k.checksum32
repository_p8w8k16m2.0.anvin_reed_sol
repository_p8w8k16m2.0_8k.checksum32
repository_p8w8k16m2.1.anvin_reed_sol
repository_p8w8k16m2.1_8k.checksum32

Trash all of these "companion" files. (Do not trash your repositoy.data file. That's where all of your data lives!)

Launch QRecall. Choose File > Repair Archive…, choose your archive, select Reindex, and Bob's your uncle.

Ralph Strauch wrote:I just tried to backup to my second archive, the one where I had the problem installing Error Correction. Qrecall now tells me that that archive needs reindexing, so I've just started a repair.

According to the log you sent me, the capture action completed but it looks like the error correction change was still running when your restarted your computer. A reindex should fix it.

Qrecall Monitor is showing Idle, while Qrecall is showing the repair in progress. I don't remember if that's standard, or not.

That's normal. Actions that modify an archive run in the archive browser window, if that's where you started it.

Ralph Strauch wrote:As I was doing this I found most of the other apps I was running had also frozen and I ended up doing a hard power down reset. I don't know whether Qrecall was the cause of all of this, or it was just coincidental.

I'm not aware of anyway that QRecall could cause your other apps to hang, so I'm going to go with "coincidence."

Ralph Strauch wrote:(Activity Monitor shows Qrecall as "not responding" and shows three instances of Qrecall Helper running, two active and one inactive. I don't want to force quit quit while the backup is running, but once it finishes I'll restart qrecall and send you a report.

It sounds like both actions might still be running. A capture runs as two instances of QRecallHelper (one to perform the capture and a second to gather user-centric metadata; the latter is usually not very busy), and the error correction task runs a single instance. So my advice is to let them run. Eventually the QRecallHelper processes will run their course and you can check the log for success, failure, or incompleteness.

Why the QRecall app itself it hung is a mystery. If you have the Activity monitor open and QRecall is still "not responding" take a sample of it and email it to me, if you have the time.

Question: Once I get this computer backed up to both archives I have a second computer to back up to both as well. Should that computer, as well, have both drives mounted when I run v2 for the first time?

It's easiest if your archive are all reachable when QRecall 2.0 is run for the first time, but it is certainly not a requirement. You can easily make these updates yourself at anytime. Here's what the "upgrade" process consists of, in a nutshell:

Replace the legacy alias structure in the action document with a modern URL bookmark. You can do this by opening each action, reselecting the archive, and saving it again.

Move any excluded items from the archive's capture action to the archive's settings. To do this yourself, simply open the archive and edit its settings to set the you items want excluded.

Move any ignored items from the archive's capture action and turn them into capture preferences for that item. To do that yourself, choose the items you want ignored in the Finder and use the new QRecall Capture Preferences service.

Finally, aliases can be finicky. Even if you let QRecall perform the 2.0 upgrade, you might have better success with your rotating set of archives if you manually open their actions, reselect the archive, and save them again. That way, you'll know QRecall is storing the correct bookmark for each archive.

Bruce Giles wrote:The standard Apple crash reporter dialog came up and I submitted it. Does Apple share any of that info with developers, or do they just use it to fix problems in their own software?

Apple shares some crash data with iOS developers, but not OS X developers. Fortunately, the crash logs are also written to ~/Library/Logs/DiagnosticReports/, and when you submit a diagnostic report in QRecall it uploads a copy of any QRecall related crash logs it finds there.

I probably did immediately re-launch QRecall and re-open the archive. If something like this should happen again (and it hasn't, so far), is there anything I can do short of rebooting to force the archive closed?

Rebooting won't have any affect on this problem.

If you have an action that's stuck "waiting to acquire access" and you are absolutely sure that the archive is not being accessed by another process, you can (a) wait 10 minutes for the arbitration logic to do its thing or (b) impatiently open the archive package and delete any invisible .lock, .shared, and .semaphore files you find:

rm /Path/To/Archive.quanta/.[ls]*

The action trying to gain access to the archive will then proceed immediately.

First, try restarting your system.

The scheduler installation is in two steps. The first defines and installs the scheduler service, and the second starts the scheduler. For some reason, the scheduler doesn't always start in OX X 10.10/10.11, or the old scheduler doesn't get stopped. Restarting will force the later to stop and should start up the former.

Whether this is successful or not, please send a diagnostic report so I can review what happened.

First, the bad news: QRecall 1.2 is not compatible with El Capitan.

The good news is that QRecall 2.0 is compatible and it has a ton of new and powerful features. The less than good news is that QRecall 2.0 hasn't been released yet.

But don't worry, QRecall 2.0 is just around the corner and you can get the beta version today. The beta version is now very stable and is nearing a final release?what's left is mostly cleanup and documentation. If you've already updated to El Capitan, you should really download the beta version and start using it.

Here's the short course on moving to QRecall 2.0:

Download the latest beta.

QRecall will first update your actions. It's best to have all of your archives on-line and reachable before installing.

Hold your scheduled actions and wait for any running actions to finish.

Open the downloaded disk image and run the Install QRecall application.

Excluded items are now a setting in the archive, not the capture action. The excluded and ignored items you have set in your capture actions will be transferred to your archive settings. (This is why the archive needs to be accessible.)

Archives created or modified with QRecall 2.0 are not compatible with QRecall 1.2. Once you have modified an archive with 2.0, QRecall 1.x will not be able to use it. (That's why you want to suspend your scheduled actions, as you might want to modify or disable some actions?possibly duplicating some archives?before you let QRecall 2.0 actions start running.)

You can browse, recall from, and verify a 1.2 archive using QRecall 2.0 without affecting its compatibility.

Review the release notes for QRecall 2.0. Detailed notes about upgrading, and most of the major new features, are explained in the release notes for 2.0.0b6.

Keep an eye out for the final release of QRecall 2.0, and please send diagnostic reports (Help > Send Report) with any problems you encounter.

Thanks for supporting QRecall!

David Ramsey wrote:Last question: will simply trying to restore data from this archive with QRecall 2 render the archive unusable by earlier versions? Or am I safe as long as I don't actually store more data into the archive?

You can continue to use the archive with versions of QRecall prior to 2.0 as long as you don't do anything to modify the archive using QRecall 2.0. This includes capturing, merging, compacting, or repairing the archive. Once you do this, the archive can't be opened in QRecall 1.2.

Browsing, recalling, and verifying the archive with version 2.0 will not change it's compatibility.