Josh,

I looked at the code for this this morning, and another thought occurred to me. It's the QRecallMonitor process (the invisible app that runs the QRecall Activity monitor window) that loads the history. Following a capture, it reloads the history for every known archive. This gives it a "bird's eye" view of what's been captured in every on-line archive. So the error might not be from either of your rotating archives. Again, you can test this by discarding all of the aliases in the Recent Captures folder, and waiting to see when the problem reoccurs.

Josh Davenport wrote:I'm getting this "? warning": Could not load capture history from /volumes/the volume/name.quanta

That's odd that you continue to get this error. Normally, a bad capture history file will be rewritten on the next capture so this doesn't repeat. Please send a diagnostic report (Help > Send Report...) so I can look for any other capture history related errors in the log.

Anything I should do?

Ignore it.

Capture history is non-essential information about the top-level items that have been captured in your archive. So, as an example, if you capture your Applications folder to an archive named MyStuff, two things happen: a tiny file named outline.index is written inside the archive package that records that this archive may contains items in your Applications folder. At the same time, an alias to that archive is created and written to <your home folder>/Library/Preferences/QRecall/Recent Captures.

Now what's all this good for? When you use the QRecall contextual menu (OS X 10.5) or the QRecall Service (OS X 10.6) to recall or recapture an item directly in the finder, QRecall scans the archives in the Recent Captures folder and quickly reads their history (outline.index) files in order to determine which archive(s) the item you've selected might be captured. So if you selected the iTunes application in the Finder and choose the "Reveal in Archive" command, the history information would tell QRecall that the archive MyStuff might contain that application.

It's superfluous information as far as the integrity of your archive or captured data is concerned. Why it's repeated failing is somewhat of a mystery, because the alias and outline.index files gets rewritten every time you complete a capture. So anything that I can image that would go wrong should get fixed after the next capture.

If you want to try to fix this, start by opening the archive package (select the archive in the Finder, control/right+click on the icon, and choose Show Package Contents). Locate and trash the outline.index file.

If that doesn't do the trick, try discarding all of the alias files in ~/Library/Preferences/QRecall/Recent Captures.

You might also consider repairing the volume that contains your archives using Disk Utility. A lot of strange file system problems (like the infamous "cannot obtain FSRef") can be caused by subtle errors in the volume's structure.

Bruce Giles wrote:Any forecast on when we'll see the new UI?

Real soon! Of course, I've been saying and thinking "real soon" for the past four months, but now I believe it's really true.

The re-imagined UI is suffering a little from the "second system syndrome" (for fans of the Mythical Man Month), where the developer throws everything, including the kitchen sink, into the next version of the product. But it's slowing coming together. A lot slower than I'd hoped, but I can see light at the end of the tunnel now.



New UI features include

icon, list, column, and a new 3D view

extensive use of animation

256x256 icons

browsing history

unified items browser (no more owners and volumes drawer)

Quick Look

X-Ray view

That wasn't the "Recall" command, that was the "Instant Recall" command, which you can get by double-clicking an item or through the menu.

Right. That's the thing I keep doing by accident, when what I really wanted to do was open a folder in the archive. (Yeah, I know, click the triangle.)

In the new UI, double clicking a folder opens the folder.

Gary K. Griffey wrote:I created a new archive...changed its settings for maximum capture compression...

Whenever the compact compression setting for an archive is increased (which will likely happen if you increased the capture compression setting), the archive gets marked as "compactable." That's because, until QRecall examines every record in the archive, it doesn't know what data could potentially be compressed. Once it's marked as "compactable," it stays that way until the compact action has had a chance to compress it.

In general, an archive gets marked as potentially needing a compact after

a merge

items are deleted

an increase in the compact compression setting

a reindex or repair

Luciano,

Thanks for posting. I agree that file format longevity is a serious issue, particularly in how it pertains to backup and archiving solutions.

Your idea of being able to export the contents of a QRecall archive directly to a ZIP archive is intriguing. I'm adding this to the wish list as something to explore for a future version. I already have plans for making QRecall archives accessible via other means, but the idea of a direct-to-ZIP export does have some merit and might not be terribly difficult to implement. It's food for thought.

Bruce Giles wrote:I'm guessing that the shaded layers were the ones added after the file had been deleted, which would mean that the search function only searches the most recent unshaded layer. Is that correct?

In a nutshell, yes. The new user interface?yes, it's still in development?will have an "X-Ray" mode that will display, and let you search for, every item that's ever been captured, regardless of whether it exists in last visible layer or not.

If so, is there a way to search all layers?

You'll have to wait for the next version.

Also, I was able to successfully recall the file, but the restore function was "grayed out", even though the preference to capture and recall using admin privileges was turned on, and the pre-authorization option was enabled.

The restore action is disabled when any earlier layers are hidden.

When you hide recent layers (using the bottom shader), you're hiding recent activity and displaying the items as they were captured in the past. When you hide earlier layers (using the top shader) you hide the history of the items and show only the changes, or deltas. Restoring items with earlier layers hidden could result in existing items being deleted, which is too scary to allow, so QRecall disables the command. For an individual file it won't make any difference, but for a package or folder the results could be quite unexpected.

So I did the recall, and of course QRecall told the Finder to open the temp folder where the file was recalled to.

That wasn't the "Recall" command, that was the "Instant Recall" command, which you can get by double-clicking an item or through the menu. Instant recall recalls the item to the temporary folder and then opens it in the Finder. It's intended as a quick, and informal, way of accessing captured items.

Why couldn't QRecall restore directly to this folder?

It can. Select the items and choose Archive > Recall. QRecall will prompt you, using a standard save dialog, for the location to recall the items to.

Or, use the much simpler method of dragging one or more items directly from the archive window to a window in the Finder.

Greg,

Thanks for the suggestion. I've added it to the to-do list.

I'll probably implement this feature by performing an unbuffered double-read of all file data, and comparing the results. If they match then everything is good. If the data doesn't match, either read it a third time to settle the dispute or log an error.

Neil Lee wrote:Besides the report I'm sending right after I post this, is there anything else I can do to track down what happened?

Neil,

The report you sent doesn't provide any illuminating information, beyond the obvious fact that QRecall successfully captured your files at 11:35PM and again at 2AM. There's nothing unusual about the number files that were captured, the amount of time the capture took, or any similar anomalies.

When anything weird happens like that on my system, I start by repairing the volume using Disk Utility or <insert your favorite disk repair program here>.

If the files were inadvertently moved or renamed, rewinding the archive to that 2AM layer and searching for those files might uncover where they went. (Unless they were moved to the trash and you don't capture your Trash.)

Gary,

Thanks for that information. I haven't encountered that particular problem, but it makes sense. The dynamic library cache contains resolved copies of the various dynamic libraries in the operation system. As a sanity check, the cache remembers the modification date and the inode (nodeID) of each library file. Following a restore, the nodeIDs of the libraries will be different, causing the OS to ignore the version in the cache.

I need to investigate this a little. Given that the files in /var/db/dyld are cache files, it might make sense for QRecall to simply exclude them from the capture and let the OS rebuild them following the restore.

The man page for update_dyld_shared_cache says

This tool is normally only run by Apple's Installer and Software Update, as they are the only official ways for OS dylibs to be updated. But if for some reason you used another mechanism to alter an OS dylib, you should manually run update_dyld_shared_cache.

Restoring a startup volume from a backup would certainly fall into this category, but it's impractical for QRecall to try and run update_dyld_shared_cache tool following a restore.

Johannes wrote:(I run without authorising because the folder only contains my stuff). I was surprised because QRecall was able to capture the file without admin privileges.

This is not surprising at all. Your user account can read this file, and all of its attributes, because the file belongs to 'admin' and file's owner has read privileges. That's why QRecall can capture the item.

When it comes to restoring the item, it can restore all of the data and the original ownership (because it still belongs to 'admin'), but it can't set the group to wheel or grant wheel read or execute privileges. That's because the wheel group doesn't belong to admin, so setting the group would escalate the privileges for this file. Without authorization, QRecall doesn't have the authority to do that.

It seems all I lost is group and world access and the original date. And I have some finderInfo attributes out of midair:

com.apple.FinderInfo:

00000000  62 7A 79 20 51 6D 52 43 00 00 00 00 00 00 00 00  |bzy QmRC........|

Now that's probably a bug, or really a side effect of not having sufficient privileges to restore the file. The 'bzy' type is a special file type that tells the Finder that the file is "busy"?that is, it's being copied or whatever?and the Finder is not supposed to let you try to move, trash, or open that item until the process is finished. Normally the 'bzy' type is removed when the rest of the file's properties are restored, but that got interrupted because of permissions.

Johannes wrote:Great! Any timeframe?

Yes, I hoped to have it done by November.

Are you still interested in feedback for the current UI? Or are things already settled for the new UI, so we better not waste time on the old stuff?

The new interface is radically different, so there's not much point critiquing it. On the other hand, if there's any features you really like about the old interface that you'd like to see preserved, speak up now.

Johannes wrote:Can't find this string anywhere in the log.

Make sure you have the log detail level control moved all the way to the right. This message is considered to be "minutia" and is only displayed at the very highest detail level.

Johannes,

Both of these features are on the way, but both require substantial changes to the user interface code—which is being rewritten right now.

Johannes,

I forgot to mention this before; if you really want to have fun with file system events, there's a secret (well, not so secret now) setting for logging FSEvent records to the QRecall log. Use the defaults command tool to set QRLogFSEvents to true:

defaults write com.qrecall.client QRLogFSEvents -boolean true

With QRLogFSEvents set, QRecall will dump all of the individual change event records that it receives from the OS. If QRLogFSEvents and QRLogCaptureDecisions are both set, it will also dump the change history tree to the log before the capture begins. This tree represents the the composite folder structure which QRecall believes contains changes based on the FSEvent records it received. Basically, any folder outside of this tree is assumed to be unchanged since the previous capture.

Richard,

I know it's hard to phrase the question, because these values are difficult to describe—but here goes:

There are three significant size numbers in QRecall. (1) The Size column in the layers pane is the total logical size of every item captured in that layer. This is, conceptually, the total amount of data contained in that layer.

(2) The size value that you see for individual items and folders in the item browser pane is the logical size of the file item, or the aggregate size of all items contained in the folder. These are, approximately, the same values you should see in the finder if you did a Get Info.

So if you capture a single 60GB file every day, both the size of the layer and the size of the item in the browser will say 60GB. Neither of these sizes reflect the amount of data that's shared with other items or layers. That's a really labor intensive value to determine, and an even more difficult value to explain, so I've never added this calculation to QRecall.

(3) To see amount of data that was duplicate, look in the log. At the end of each capture, the log records the logical size of the items that were captured—this becomes the size you see in the layers pane. If you expand that log line (you might need to slide you details control over to the right), you'll see a breakdown like the one shown in the attachment. This will tell you how much of that 60GB is new data and how much was duplicate.

I hope that helps.