Just dropping a quick note re: 1.1.0(9) beta.

All seems to be working fine. Beta 9 also fixed the "no free space" problem I was having too.

The use of Leopards FS change services (spotlight) to find changed files works very well. Looking through the captured layers I find files I know changed being captured.

Only question I have (and I don't think it's beta related) is how long it takes a capture to "locate free space".

I'm backing up two machines to the same archive. At first I thought the locating was fast when the same machine captured to the archive without the other in between (Capture A, Capture A; not Capture A, Capture B, Capture A).

But I think an intervening Merge/Compact has more impact; but I'm not sure.

I'm backing up to an archive over AFS. Using both ethernet and WiFi. Of course the wifi is slower; but works.

Is there anyway you could index the free space instead of "locating" it? Also could you add to the log the amount of time spent creating the scribbles, and locating the free space (to the normal logs; not #debug).

I've attached a section of the log from a recent capture of the Wifi machine.

As always if you need more information or need me to perform something, just ask.

Thanks

Rodd Zurcher

Rodd Zurcher wrote:Only question I have (and I don't think it's beta related) is how long it takes a capture to "locate free space".

I'm backing up two machines to the same archive. At first I thought the locating was fast when the same machine captured to the archive without the other in between (Capture A, Capture A; not Capture A, Capture B, Capture A).

But I think an intervening Merge/Compact has more impact; but I'm not sure.

It has everything to do with the Merge.

A merge removes two or more layers and replaces it with a composite of those layers. When its done, it may have removed items that may, or may not, share common data with other files still in the archive.

The "locating free space" phase will occur on the next capture or compact following a merge. What QRecall is doing is something called "mark and sweep" in computer jargon. It first "marks" all of the quanta that are used by at least one file and then "sweeps" all unmarked quanta into the trash bin.

I'm backing up to an archive over AFS. Using both ethernet and WiFi. Of course the wifi is slower; but works.

How long this takes depends primarily on the total number of items records in the archive and fast it can read them. Reading millions of file records over a WiFi connection will take awhile.

You can mitigate this by merging less often and only on the machine with the fastest connection.

Schedule a merge followed by a compact on the one computer with the fastest access to the archive and remove the merge and compact actions from all other computers using that archive. If the AFS server is a computer (not a Time Capsule or other NAS), install QRecall and schedule the merge and compact on that computer. You do not need an identity key to schedule and run maintenance actions.

If you have the disk space to spare, consider merging only once a week or add a condition to the merge action so that it only runs when the free disk space falls below some reasonable margin.

Is there anyway you could index the free space instead of "locating" it?

That's exactly what it is doing. It's "locating" the unused data so it can add it to the index of free space.

Also could you add to the log the amount of time spent creating the scribbles, and locating the free space (to the normal logs; not #debug).

That's a good idea. I'll add some message indicating how long it took to locate the free space. I might even be able to log the amount of free space it found. The timing of scribble files, however, is somewhat meaningless as this happens in parallel with the capture. It's only an issue if the capture finished before the initial copy does.

Thanks once again for your detailed explanation.

I already have the ethernet connected machine doing the merge. But based on your advice I'll use the ignore if free-space condition, and run a compact immediately following the merge.

Thanks

rbz

Another followup.

After delaying merge's to once a week, and following it with an immediate compact, my backups via the wifi machine have dropped to ~7m when Leopard file change detection is used; and ~45m when QRecall does a deep file change scan.

This is wonderful.

Great work.

rbz