I'm currently in the process of creating an archive of around 15GB to an external USB2 drive (Western Digital Passport 160GB).

Speed: around 95Mb per minute (after 1hr 40mins and 12.5GB)


Processs CPU mem VM
Monitor 2% 2MB 350MB
Helper 10% 310MB 950MB
QRecall 3% 4MB 380MB

System reports 600MB of Inactive memory.

System is quite slow and unresponsive (iMac24" - 2.16GHz Core 2 Duo 2GB Ram) for other applications.

I notice that QRecall seems to slow down considerably when processing larger files (10MB and up).


Any thoughts?

This is an unfortunate consequence of the amount of work that QRecall is doing. The QRecallHelper is that task that's doing the lion's share of the work. It uses a lot of RAM and can saturate the I/O at the same time.

If you are trying to use other applications at the same time, the problem is probably virtual memory swapping. While QRecall is working it pushes segments of code and data from other applications out of memory and into the swap files. When those applications need that code or data they have to swap it back in. This is made more difficult because the disk drive is already busy copying file data to the archive, so those swap requests have to wait.

You can verify this by watching the page in/out numbers in the Activity monitor. Sluggish behavior will be accompanied by ever rising page in/out numbers.

You can reduce the amount of work that QRecall does by lowering the Shifted Quanta detection setting for the archive. (Setting at it lowest setting essentially turns it off). But this won't reduce the burden on I/O. In fact, it will tend to increase I/O.

I have plans to improve the performance of QRecall in several places. This requires restructuring some of the code to take better advantage of multiple processor systems and reduce the amount of memory it uses. Both of these should improve how QRecall plays with others.

Unfortunately, these changes require significant modifications to the core capture routines -- changes which must be made very carefully. So I don't have plans to make these performance improvements in the immediate future. But if performance becomes a persistent problem for people, I can re-prioritize that work.

I understand that QRecall is doing a lot of work here. I guess my particular queries were:

I seemed to have 600MB of inactive memory, so the memory wasn't an issue.

The speed would go up to 400MB/min for smaller files, but seemed to slow considerably for larger files. I would have expected the opposite.

I've been a longtime user of Chronosync - it seems to "play better" with other applications.

I guess when creating the initial archive, its best practice to set it going overnight or something. I note that the recapture was by comparison fast and had much less effect on the rest of the system.

john hampson wrote:I seemed to have 600MB of inactive memory, so the memory wasn't an issue.

This is a common misconception. Inactive memory is not unused memory. It's just memory that hasn't been accessed recently (read "few seconds").

The speed would go up to 400MB/min for smaller files, but seemed to slow considerably for larger files. I would have expected the opposite.

I've been a longtime user of Chronosync - it seems to "play better" with other applications.

This is because of the fundamental difference between QRecall and other backup applications. QRecall analyses every block of data in a file to determine if any of that data has already been captured (in that file or any other file). It also searches for duplicate data that has shifted to a different location in the file.

By contrast, applications like Chronosync, SuperDuper, Retrospect, et. al. merely look for files that have changed and copy those files. The larger the file, the less work they have to do.

For QRecall, the larger the file the more work is has to do.

Interrestingly, QRecall is more of a resource hog on my Intel Core 2 Duo macbook than on my old iMac G5 ?

It *is* annoying on the macbook to work alongside a backup.

Fred

Most performance degradation is due to swapping. The amount of RAM you have can have much more impact on performance than CPU or even drive speed.

The macbook has 2GB, the iMac 1GB...

The thing is, you can feel the macbook unresponsive for a second, and I don't feel that with the iMac.

On my MacBook Pro I do notice that the hard drive is significantly slower than in my desktop machines. A few applications that fly on my G5 seem to crawl on my MBP, even though my MPB has more (CPU) horsepower and just as much RAM. When I put my ear to the MBP, I can hear the drive just trashing about.

I haven't really noticed any slowdown with QRecall because I run my captures at night.