Adrian Chapman wrote:... there is an unknown damaged layer that I don't seem to be able to remove. although it doesn't seem to be doing any harm I'd like to be rid of it if possible. Any ideas?

A "-Damaged-" layer means that one or more of the items that were originally captured in that layer has been lost or damaged.

To determine exactly what was damaged you'll need to refer to the log or examine the layer. The repair command will log everything it finds wrong with the archive and what it did about it. More than likely, your archive contains an invalid block of data (disturbingly common when capturing via WiFi) belonging to a file. The repair will mark that file as "damaged", or delete it altogether, depending on the amount of data lost.

As an alternative to the log, shade (hide) every layer except the "-Damaged-" one, and then explore its contents. The items that were damaged will also be marked as "damaged" or "incomplete."

It might be possible get rid of your damaged layer by merging it with subsequent layers. If the damaged item, or items, was successfully captured in a later layer, merging it with that layer will replaced the damaged (or missing) item with the successfully captured one. The resulting merged layer will be complete again. On the other hand, if the next layer did not recapture that item (because it didn't changed), the merged layer will as be marked as damaged.

So it's possible to expunge your archive of damaged layers by merging, but you'll also lose all of the other intermediate changes that occurred between those layers.

My advice is to simply be aware of the problem and be wary of damaged items when recalling from that layer.

The damaged layer should eventually be erased through the normally scheduled merge actions. Also note that a file or folder marked as "damaged" in the repository forces QRecall to recapture that item during the next capture. Thus, a capture following a repair will recapture any possibly lost items in the archive.

Adrian Chapman wrote:To keep the backup processes fairly slick on my Mac Pro, I perform a full backup of the boot volume once a day and 2 hourly backups of my user directory, but I have excluded three large folders from my user directory which only change infrequently, namely Music, Pictures and TV Programmes. These last three folders are backed up just once per day to their own archives. The effect of this is that my full volume archive is now more manageable and so connecting to it is relatively quick, important for the 2 hourly backups, it was taking about 5 minutes to connect when everything was in it.

Adrian,

Your approach is sound. I do have one question: Are you running the release version (1.1.4) or the current beta?

I ask because the beta has significant performance improvements, particularly in the amount of time it takes to add relatively small captures to an archive. If you're not using the beta, you might give it a try and see if you can include your music and video files with acceptable performance. (Remember that you can always delete those folders from your archive if you want to go back to your two archive arrangement.)

Regardless, what you have set up seems perfect. I might also suggest scheduling a verify action (again, on the MacMini), that runs about once a week. It's good to periodically check the integrity of the entire archive.

Gary K. Griffey wrote:Possibly, in a future release...QRecall could provide the ability to override the use of the file system events via preferences...

I will add that to the wish list.

Gary K. Griffey wrote:This does, however, raise some rather disturbing questions in my mind...most notably...is this happening with other QRecall captures that I rely upon...and am I simply not aware of it?

It's entirely possible. If you are regularly moving source volumes from one system to another, then any software that relies on file system events to detect changes on those volumes should be treated with suspicion. This issue will also impact users who maintain multiple instances of the operating system (often for testing) and who regularly reboot their system from alternate volumes.

These are, however, uncommon scenarios so it isn't a problem for most users. Most of the volumes/devices uses to store documents—the kind of volumes that you would capture to an archive using QRecall—do not get passed around from one system to another. For volumes that are occasionally shuffled between systems, the 7 day setting of QRAuditFileSystemHistoryDays insures that they will get captured correctly sometime in the next few days.

The volumes that are regularly shared between systems are more likely be used to backup to (rather than from). The volume containing the archive is immune to this issue.

The best I can recommend is to be aware of the limitations of OS X's file system events log; if it's a problem for QRecall, use the QRAuditFileSystemHistoryDays setting to limit its impact or ignore it altogether.

(This should probably should be a new topic, but JForum won't let me split an existing thread....)

Gary K. Griffey wrote:This morning...I performed the exact same procedure...I restarted the laptop in target disk mode...attached it to my second laptop using a FireWire cable...and attempted a QRecall recapture of the entire drive...the recapture finished in about 10 seconds...

I suspect that you've been bitten by the file system events (a.k.a. FSEvents) service. I'll quote from the Advanced QRecall Settings page:

Leopard's folder change detection is not foolproof. There are a number of obscure situations where the file system will not accurately report the changes on a volume.

One situation where it isn't foolproof is (drumroll) moving a drive between different systems—especially if those systems aren't running the exact same version of the OS.

What happens is that the volume's file system change log gets reset, and when QRecall queries the volume for changes it comes back with nothing (or very little), and QRecall skips capturing items that have, in fact, changed since they were last captured. You can verify this by looking in your QRecall log. Open the QRecall log window and slide the details control all the way to the right. In the log messages for the capture you'll find something like this:

Locating changes since Tuesday, August 3, 2010 7:30 PM

Collected 117,752 folder changes

If the number of changed folders was zero or very small, then the operating system has lost the history of changes for that volume.

QRecall knows that the file system change log isn't reliable and will periodically ignore it. Unfortunately, the default period is about a week:

To guard against this, QRecall only trusts the operating system for a limited amount of time. After that (approximately 7 days) the capture will ignore the system and perform a deep, exhaustive, scan of the entire directory structure looking for changes. Once the deep scan is complete, QRecall will again trust the operating system's change detection for another 7 days.

You can work around this problem be reducing the amount of time QRecall trusts a volume, or disabling the feature altogether, by setting the QRAuditFileSystemHistoryDays advanced settings. Put your MBPro into target disk mode, plug it into your other system, open a Terminal window, and issue the command:

defaults write com.qrecall.client QRAuditFileSystemHistoryDays -float 0.0

This will completely disable QRecall's use of the file system change events log. It will, instead, check every file and folder for changes on every capture. Note that this can significantly increase the amount of work QRecall does on each capture.

Capture your MBPro. QRecall should find and capture all changes on the volume. When you're done, restore the default be deleting your custom setting. Again in Terminal, issue the command:

defaults delete com.qrecall.client QRAuditFileSystemHistoryDays

In the future, you have the option of repeating these steps each time you capture your MBPro or you could leave QRecall's "trust" period set to something much smaller than its default value of 6.9 days. For example, setting it to 0.9 would mean that a second capture that's more than 22 hours after the previous capture would trigger QRecall to perform an exhaustive scan for changes.

Gary,

Thanks for the feedback. It's always good to see real-world numbers—especially when those numbers show improvement.

Wayne Fruhwald wrote:After updating my production copy of Qrecall to the latest beta (1.2.0 Beta 6) one of my archives got corrupted. I'm not sure if it is directly related or just a coincidence.

Your guess is as good as mine.

During the required "repair" operation I have received multiple "Transient data read error; re-reading the same data was successful" errors.

When QRecall reads a block of data and finds it to be corrupted, it requests a second (unbuffered) read from the drive to a different RAM location. If the second read returns uncorrupted data, it logs the message you are seeing and continues.

This message means that the data on the drive is, more than likely, OK and that the data is being corrupted somewhere "in the pipe." The data could have been mis-read from the media, scrambled during transfer through the interface (USB/Firewire/eSATA), or could have been dinged by bad RAM.

If this message persists, I would suggest a thorough RAM diagnostic. This wouldn't be the first time that bad data was caused by failing RAM. I would also try to test the drive using a different interface (i.e. switch from Firewire to USB), if that's an option, and see if the errors go away. It's also instructive to notice if the file location associated with the error changes. If the problem persists at the same location, it would point to a media problem. If it changes, then it would indicate transient corruption elsewhere.

Do you automatically re-write the data to a new location on disk to prevent future transient data read errors? If not, is there an option to force you to as those areas on disk are not most likely to go bad so I would like to have the data relocated to a safer place.

Sector sparing is now the purview of modern drive controllers. This should be done automatically by the hard dirsk controller, and it's no longer possible to do this from software—which is a good thing, if you ask me.

It should be noted that the error you are seeing occurs while reading (not writing) the data. The fact that the data can be re-read means that it's probably stored on the magnetic media OK and that the problem is in the transportation of the data, not its storage. Detecting bad sectors on write, sparing those sectors, and writing the data to a more reliable region of the drive is the job of the hard disk controller. If it's not doing that, then you need a new drive/controller.

Iain Farquhar wrote:Recalled my desktop (happens to be quite a large amount of data as I've got an email archive there)

Dismayed to find that all the data is copied back to my drive in a location as follows.

MBP HD:private:tmp:InstantRecall.501:QRecall-30160298357esktop

That's QRecall's "instant recall" feature. It reassembles a file from the archive into the system's temporary directory (/private/tmp) and opens it, allowing you quick access to items in your archive. It isn't intended as a way of browsing a large number of items.

I've quit QRecall but the file stays there.

Restart your computer or wait 3 days. The system periodically deletes items in the temporary folder that haven't been accessed in awhile, or whenever the system is restarted.

I would like to look at the backups without copying them back to my drive, does QRecall do this?

That's what the archive window is for.

I would really like a more sophisticated way to navigate my backup without copying all the files back to my boot drive.

A completely rewritten archive browser is currently under development and will be debuted shortly as a beta. If you'd like to try out the new browser and provide feedback and suggestions during development, I encourage you to download and install the beta at www.qrecall.com/download.

Christian Roth wrote:is there a way to (globally, at least on one machine) put any scheduled operations on-hold?

Not at this time, but it's on the to-do list for 1.2.

David Cretney wrote:Assuming I do a complete volume back up using qrecall, wipe the drive, and install snow leopard I am still faced with a manual restore and re-install scenario right?

Let me suggest an alternative:

- Don't wipe your drive.

- Make a backup (this is your safety net, but not your migration tool) using QRecall.

- Now install Snow Leopard, choosing the "Archive and Install" option. This option takes all non-Apple system-level components and sets them aside (in a disk image you can later peruse). The installer then installs a fresh, completely clean, copy of Snow Leopard but preserve your users, their home folders, any third-party applications you had installed, your system network configurations, and so on.

This is conceptually equivalent to installing a new copy of Snow Leopard, creating user accounts with the same names and UIDs as you had, then restoring all the home folders, Applications, Fonts, etc. that you had before, reconfiguring everything in System Configuration, and so on. Yes, you could do this by hand, but the Archive and Install option is much easier.

If anything goes horribly wrong, you've got your backup fall back on. If you really want to do it by hand, you may have ownership and permission issues to deal with.

Chris,

Sometimes getting the background components installed and running can be a bit tricky.

Try this:

Open QRecall

In the preferences (QRecall > Preferences, Authorization tab), make sure is says that QRecall is authorized to use administrative privileges

Uninstall QRecall (Holding down the Shift and Option keys, choose QRecall > Quit and Uninstall)

Restart the computer

Launch QRecall

In the preferences (QRecall > Preferences, Authorization tab), Preauthorize QRecall to use administrative privileges

Restart the computer a second time

Launch QRecall and send a diagnostic report (Help > Send report)

That should fix it. If not, the details of the problem should be in the report.

Glenn,

This is a bug in Launch Services. I've been bumping into this since the 10.4 days.

The problem is that "something" happens to the Launch Services database that causes QRecall to crash when it asks the operating system for the icon of a (usually hidden) file.

This can sometimes be fixed by simply getting the finder to visit the item and refresh it icon information. If you know, approximately, where QRecall crashes simply open up every folder and subfolder at or near that location. This causes the Finder to reload the icon information for each displayed item. For some reason, this usually fixes it.

If you have no idea where QRecall is crashing, or just don't care, you can usually fix the problem by resetting the Launch Services database with this ungainly command in the Terminal (all one line):

/System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/Support/lsregister -kill -r -domain local -domain system -domain user

Restart and try it again.

Before you do this, however, also make sure that there's nothing wrong with the directory structure of the value you're capturing from. Use Disk Utility to perform a verify of the volume. Repair it if necessary before proceeding.

You can also send in crash reports using the Help > Send Report... command. This also includes some anonymous system profile information and your recent log records, which can help diagnose the problem.

Thanks to everyone who sent in diagnostic reports!

Problem solved!

As it turned out, there was nothing wrong with the archives at all. Under a rare set of circumstances, the verify action would occasionally count the same quanta hash index entry twice, resulting in an erroneous total.

So if you encounter this error in 1.2 beta 5 or earlier, just ignore it—your archive is fine.

The fix for the verify action will appear in the next release.

Mark B Anstendig wrote:I am not interested in archiving. I just want to create a schedule to do around 5 separate copies of 5 hard drives containing my historic photograpgs (see www.anstendig.com) and to do them simultaneously. The copies will be, of course, on another 5 drives.

QRecall does not copy drives. It's an advanced backup and document archiving system that analyses the data in a large collection of files and incrementally captures only unique data that has changed in a compact archive.

If you just want to copy five drives worth of data, everything you need is in the OS. iCal and few AppleScripts is all you need.

Personally, I think you're wasting a lot of CPU time and bandwidth with that approach, and you're gaining very little in the way of data integrity. A RAID would offer better security (and possibly better performance). QRecall would be a vastly more intelligent, efficient, and secure method of preserving your data.

Also, does QRecall work with FIS based port
multiplier enclosures. FIRMTEK contacted me and thinks Retrospect cannot do simultaneous actions on their enclosures because Ret. has a problem working with that type of FIS based enclosures.

QRecall will work with any volume you can mount in OS X.

QRecall's principle restriction is that it will only allow one action to update an archive at a time. Beyond that, you can create as many archives as you want and run as many simultaneous actions as you want.

What I was basically saying in my earlier post is that QRecall will let you start as many actions as you like—you could perversely schedule a hundred actions to run at once—it's just unlikely to be very efficient.

I encourage you to experiment and report your results. Schedule five actions to run at the same time and set the scheduler to allow unlimited number of concurrent actions. Then drop it down to 4, 3, 2, and 1 and see what gives you the best performance.