Christian Roth wrote:I am seeing the issue that after what looks like a mostly finished Reindex of my archive, the process stalls in the "Reading layers" stage. The QRecallHelper application is still actively doing something, and I checked using Instruments that it is alternatingly reading and writing chunks of 12 bytes in size to a single file.

QRecall is updating the quanta hash. The "Reading layers" message is an artifact. It just happened to be the last progress message that got put up before the reindex finishes. Just before the reindex command begins the process of closing the archive it cleans up the quanta index (mostly to get back memory), which is where it appears to be stuck. I'll make a note to insert an additional status message in there to so it's more obvious what's really happening.

If QRecallHelper is reading and writing 12 byte records, then it's probably doing what it's supposed to be doing, which is writing all cached hash records to the hash.index file. The hash.index file never changes size. It's the largest data structure in a QRecall archive and consists mostly of a huge array (i.e. tens of millions) of 12 byte records. It allows QRecall to quickly find any quanta in the database.

Normally, flushing the records to the hash doesn't take more than a few minutes. However, it can be influenced greatly by the CPU, amount of RAM, archive access speed, competing processes, etc. I have a MacMini that captures to a 1TB archive that can get stuck updating its quanta index for hours (in fact, it's upstairs doing that right now).

Is there any actual progress even being made? What is getting read (and written back?) to hash.index at this stage,

Given that you're seeing the QRecallHelper process continue to read and write 12 byte records, I'm pretty confident it isn't stuck. However, I've been wrong before.

and why is it taking so long?

That's a complex question with a lot of variables. In my experience, one of the biggest factors is speed of access to the archive. If the archive is on a networked volume or USB connection, the overhead of reading and writing lots of tiny records can be high which can dramatically slow the process of updating the hash.

Hopefully by the time you read this QRecall has finished and moved on. In the unlikely case that it really is "stuck," take a sample of the QRecallHelper process and send it to me along with a diagnostic report (Help > Send Report...). You can obtain the process sample in Activity Monitor by locating the running QRecallHelper process and clicking on Sample Process, then saving the results to a text file. If you're a command-line fan, you can use the 'sample' tool to do the same thing.

One last question. What prompted you to reindex the archive in the first place?

Mark Gerber wrote:Do the claims of space-saving efficiency apply to graphics files, too?

In the interest of full disclosure, I have to say "it depends," but in your case the answer is "yes."

For instance, if I add a layer or two to a 300 MB Photoshop or Painter file, will QRecall only back up those layers so that I don't end up
with two 300 MB files?
Or if I make changes to an existing layer, are only those changes added to the back up file?

This is exactly why I wrote QRecall. In fact, Photoshop documents were used as the first test files for QRecall.

Photoshop and similar graphics applications tend to write the layer data sequentially in the document file. Inserting a layer tends to just "push" the data in the other layers to a new position in the document.

QRecall can detect this "shifted data," but by default it doesn't look for it. If you try QRecall, make sure you bump the "Shifted Quanta Detection" in the Archive Settings up a notch or two.

Is it practical to have QRecall back up these as files as I'm working?

Yes. Photoshop and similar applications write documents in their entirely when you save them, so the contents of the documents aren't in flux while you work. This means that saved documents are captured completely. The only potential problem would be if QRecall was capturing an item at the same instant that you were saving it. Regardless, the next capture would absolutely capture the current version.

When I work with Painter and, because I have had reason in the past not to trust their native RIFF format, I typically save iterative versions of a working file. At the end of the day, I might have an additional five or ten or fifteen saved files. I'll usually delete all but the last three or four and keep those until the next day when I start again, repeating this until the project is finished (which can be anywhere from one to six weeks after it's started), at which time the final version is archived to DVD and the iterative versions deleted.

So it might look like this:

(clip) QRecall won't have any problem with that at all. It will see most of these files as semi-duplicates of each other and store only one copy of the data.

I'm trying to figure out a back up strategy that will give me the protection I need without filling up a drive too quickly.

Given your work flow, I'd say that QRecall is perfectly suited.

But don't take my word for it. Try QRecall and see how it works for you. After using it for awhile, check the log and look for the capture action details. The log will record how efficiently QRecall is detecting duplicate data.

For the best performance, set up two archives. One archive to backup your whole system. Have that archive capture the entire startup volume every night. Then set up a second archive just for your working project files. The second one can be set up to capture repeatedly during the day. The small size of the archive and capture target ensure maximum performance a minimum interference with your work. And remember to set the Shifted Quanta Detection on the project archive.

This is exactly how I have QRecall configured. One archive backs up my entire development system once a day, and a second archive is set up to capture my QRecall project files every 20 minutes between 6AM an 9PM.

To Manually Uninstall QRecall:
- Stop all running actions and Quit the QRecall application.
- Delete the QRecallMonitor Login Item from your account preferences (Mac OS X 10.4 only)
- Delete any files beginning with com.qrecall from the /Library/LaunchDaemons, /Library/LaunchAgents, and/or ~/Library/LaunchAgents folders.
- Restart your computer.
- Delete the /Library/Application Support/QRecall and/or the ~/Library/Application Support/QRecall folders.
- Delete all files in ~/Library/Preferences that have names beginning with com.qrecall.
- Delete the ~/Library/Preferences/QRecall folder.
- Delete the ~/Library/Contextual Menu Items/QRecall CM plugin item.
- Delete the QRecall application.

Mark Gerber wrote:Will QRecall play nice with itself or do I need to make sure the schedules for each of the computers don't overlap? I'd guess that's less a concern when it's writing to three different archives, but what about if I choose to add licenses so they are writing to one?

Using separate archives simultaneously is no problem. You may encounter some slow down as multiple systems vie for drive and network bandwidth.

QRecall will arbitrate between multiple actions or systems trying to use a single archive at once. The other scheduled actions will simply wait until the first one is done.

Mark Gerber wrote:I plan to get an external drive that will be used for our back up files--I guess this can be something like Apple's Time Capsule (is this an NAS drive?) or one that can be attached to one of the computers and accessed by all through the network.

Yes, the Time Capsule is a type of NAS drive. A NAS drive or a drive on one system that you share would be equivalent.

Would I need QRecall running on each computer and then on each user's account (as many as six copies) to back up everything to this external drive?

That's not necessary. If you install QRecall in an administrative account on each computer and then pre-authorizing it to use administrative privileges, it will be able to capture all of the files for all of the users on that system.

If I understand correctly, I can use one license for all our back up needs, but that creates one database for each user (or more in this case?).

Reusing the same identity key for each system, create a separate archive for each system. The one installed copy of QRecall on each system would capture everything on that machine to its own archive.

To make the most efficient use of the space on the hard drive, how many licenses would you recommend?

To take full advantage of QRecall's ability to merge duplicate data (multiple copies of the operating system, applications, music, ...) you need to capture all three systems to a single archive. That would require three identity keys, so that QRecall can keep the files from all three systems separate in the archive.

I'd suggest you just start with a single identity key, capture the three systems to three different archives, and see how it goes. If you start to run out of disk space you can always turn on compression and/or buy additional licenses to share a single archive later.

Steve Mayer wrote:I've got OSX configured to have double arrows at the top and bottom of the scroll bar.

That's probably it. You are correct, if the scroll bar region gets too small to fit the scroll bar control simply disappears. I thought I had set the minimum size of the layer pane large enough to prevent this, but I didn't consider double arrows at the top and bottom of the scroll bar.

Kevin Crawley wrote:What if I were to restore the files from QRecall running on the cablecast machine?

Oh, then the cablecast machine is another Mac? (I was thinking it was some Linux box or something you used for video recording.) Yes, that would work. In fact if it's a Mac, the best thing to do would be to set up QRecall on that machine and capture and recall directly.

Will I need a new identity key for the cablecast machine for that purpose?

Only capture requires an identity key. See Identity Keys.

If you want to set up QRecall on the cablecast machine to capture, you have two choices. You can reused your existing identity key under the terms of the household license and capture to a separate archive. If you want to capture to the same archive, you should get a second identity key so that QRecall keeps the files in the archive separate.

Try resizing the window and moving the separator bar between the layer browser and the item browser.

If that doesn't fix it, try this:

- Choose Archive > Settings... and make note of your settings.
- Close the archive window
- In the Finder, right/control+click on the archive and choose Show Package Contents
- Drag the settings.plist file out of the archive package. E-mail me a copy.
- Reopen the archive in QRecall. See if your scroll bar has returned.
- Choose Archive > Settings... and restore any archive settings that were forgotten.

I'm assuming that you are accessing the files on the cablecast machine via some sort of file sharing.

The problem is probably permissions. When you access files on local drives, the operating system restricts access based on your logged in user ID. You are limited to what files you have access to and what properties you can set. For example, you can't change the ownership of a file that you don't own.

When you pre-authorize QRecall to use administrative privileges, it lets QRecall run as root which gives it carte blanche access to read and write any file or permission set on your local volumes.

For network volumes the same rules generally apply, but the access given is granted by the remote file service not the local operating system. So it doesn't matter if you run QRecall as root or not, what privileges you have on the remote volume are determined by how you logged in to the remote service.

I suspect that you connected to the comcast box using a user that is different than the one that owns to the multimedia files you captured. Given read access to that directory, QRecall will capture the owner and permissions of those files, but since it isn't the owner it can't restore the ownership and permissions back the way they were.

The only solution I can think of will require that you gain access to that volume in such a way so that you have permission to restore the original ownership and permissions of those files. If you can attach to comcast volume using the owner of those files, that might do it. You might also physically extract the hard drive from the device and attach it to your Macintosh as an external drive.

Got home last night, bought a key this morning, and then started a capture by hand

The problem is that the capture seems to be starting from scratch all over again

Yes, the capture is starting over. The reason is because you're using a new identity key.

QRecall now sees your system as a new owner and (from its perspective) is capturing everything for the first time. As it does this, it notices that every one of your files is remarkably similar to the ones captured by that "other" user who had the trial identity key. So the archive isn't growing substantially, but it will create a new layer belonging to your new identity that contains every item on your volume.

When its done, you'll have two owners in your archive. If you want to recall something from before the identity key change you'll need to switch to the old owner in the Owners & Volumes drawer. Eventually you'll probably just want to delete that owner by selecting the volume belonging to the old owner and choosing the Archive > Delete Item command.

I can't think of any reason why QRecall should interfere with Time Machine. The error associated with QRecall's Preferences folder is especially mysterious, since there's nothing special about that directory at all. In fact, unless you edit an action or an archive is moved, the contents of that directory don't change.

Maybe Apple updated Time Machine to stop working if QRecall is installed.

I suspect that something else went wrong. I have no ideal what error 11 is, and suspect you'll never find out as Apple doesn't publish any Time Machine error code specifics. There's a couple of threads on Apple's discussion forum where people have reported failed Time Machine backups with error -43, but they don't appear to have garnered any response from Apple.

The general consensus of these threads seems to be that "Time Machine is great, just make sure you have a second backup strategy."

I can suggest a few ways of doing this.

The manual method, equivalent to pointing Time Machine at a new drive, is to do that same with QRecall.

- Disconnect the old drive, plug in the new one
- Create a new archive (or copy the one from the old drive)
- Open any action that used the old archive and select the new archive
- Save the action

If you rotate backup drives on a regular basis you could automate this to some degree by creating two sets of actions, one that uses archive A and an identical set that uses archive B. In each action, add an Ignore If No Archive condition.

As long as the drive with archive A is plugged in and mounted, the A set of actions will run. As long as the drive with archive B is mounted, the B set of actions will run.

A slight variation on the second approach is to create a set of actions that run when a particular volume is mounted using the Archive Volume Connects event schedule. Again, create two sets of actions for archive A and archive B. Now, whenever you connect the volume that contains archive A, the A set of actions runs immediately. When you connect the volume containing archive B, then B set of actions start.

I have, but just haven't gotten to it. I'm developing some short videos right now to help explain how QRecall works and how it's different from other backup utilities in general. But I'm sure a feature by feature comparison would be helpful too.

I'm still surprised at the number of people who think they're protected with a clone backup, That is, until they need to recover a deleted or damaged file *after* they've already synced to the clone...

It's truly shocking.

Bruce Giles wrote:There's one thing about QRecall's user interface that bugs me a little. When I'm in the Finder, in a list view window, and I double-click on a folder, the folder opens. When I do the same thing in a QRecall archive window, because old habits die hard, it immediately does a recall of the double-clicked folder.

You are not the first to complain about this particular "feature." I plan to address this issue in the next release.

Usually, that isn't what I wanted to happen. What I wanted was to see what was inside the folder. I wish QRecall had a preference that would let me decide what it should do when I double-click something in the archive. At a minimum, it would be good if it could act as if I had clicked the disclosure triangle.

That's a good suggestion. I've also considered disabling the double-click action so you would be forced to use the Action > Recall and Open command explicitly.

When I do accidentally recall a folder that way, I see that QRecall puts it in a subfolder of /private/tmp. If I remember correctly, that folder is automatically cleared on startup (or maybe shutdown?).

/private/tmp gets cleared on restart and is also (normally) scanned once a day for items that have not been accessed in the last 3 days. (see /etc/defaults/periodic.conf and /etc/periodic/daily/110.clean-tmps).

Does that mean I don't need to worry about explicitly deleting accidental recalls, as long as I don't mind them hanging around until I reboot?

Yes, which is exactly why they get recalled to /tmp.

Thanks for the confirmation, Bruce.

I'll be out of town next week. When I return, it looks like rc2 will become QRecall 1.1.

Thanks again to all my beta testers for your help and feedback. I couldn't have done this without you.