Something is seriously wrong, but I?m not sure what it is.

If I had to guess, I?d say you had volume structure corruption. You said you checked the drive, but I want to make sure you checked the volume the archive is on (not the volume you were capturing).

The initial error was MacOS error -47 (file busy). This makes no sense on a file that?s on a local drive and is already open. Then, your repair actions fail because the stale .lock file can?t be deleted. The error returned was 22. This is either a dsNoPk5 (no package 5) error or EINVAL (invalid argument). Again, neither of these errors make any sense unless something internal to OS X is botched up.

My suggestions would be (a) restart the system, (b) run disk repair on the volume containing the archive, and (c) make sure you are NOT running any anti-virus, disk optimization, or disk virtualization software that might be interfering with QRecall?s ability to read, write, or delete files on that volume. If your external drive is using ownership and permissions, make sure the files belong to your QRecall user and have read and write access. If they don?t, either fix that or turn ownership and permissions off for that drive.

Then try to repair the archive. Repair should delete the .lock file and verify/restore the integrity of the data in the archive.

Mats,

It?s difficult to guess what?s going on here. If you haven?t already, send a diagnostic report (Help > Send Report).

The archive I/O error would indicate a (possibly transient) drive failure. The fact the disk utility give the volume a clean bill of health is excellent news, and your read test of the archive.quanta file is also excellent. While it doesn?t test the writability of the file, these are all good signs and would indicate (to me) that this was a transient error, probably a bus failure.

The .Lock file is very strange. Is this a local (directly connected) volume? If so, I can?t see anything that would prevent a sudo rm /Volumes/Safe House/Dagligen/dagligdags.quanta/.lock command from failing, beyond something seriously wrong with the operating system or the volume structure. If this is networked volume, then sudo will have no effect and you?ll need to delete the .Lock file (or just repair the archive, which will delete the .Lock file) on the computer that?s directly connect to the volume.

I?d be able to look into the further with a diagnostic report.

Norbert Karls wrote:? why is NAS over WiFi (even 800MBit/s) so excruciatingly slow in comparison with that very same NAS over CAT-5, FW-800 or USB-3.0?

This is the nature of WiFi. While the raw transmission speeds of the WiFi and Eithernet have gotten remarkably close, the reality is that the two operate in completely different environments. WiFi has to deal with interference, signal strength, collision avoidance (CSMA/CA vs. CSMA/CD), competing WiFi transmitters, and the latency of radio signals. While there are often things you can do to improve your WiFi performance?like switching channels?most of the performance issues are physical limitations of trying to conduct TCP/IP transactions using a radio.

Can we do anything to improve QRecall performance when backing up via WiFi? Maybe downloading some index data just once and using it locally?

I?ve considered local caching of some index files, but most of the indexes are already read once and cached in memory (which is quite fast, even using WiFi). The biggest problem is latency and the random accesses needed to match the data being captured with the data already in the archive. Random access really take a beating (performance wise) when latency is high.

For equipment that doesn?t wander around, Ethernet is much preferred over WiFi.

As for your merge action and schedule, this is entirely a matter of taste, need, and resources. Your archive isn?t that big, so you?re not constrained by resources—either by your disk space limits or computer time. So the only question is how much information do you want to capture and for how long.

The questions I like to ask are ?How often do my files change?? and ?How long would it be before I notice a problem I?d want to recover from?? Answering those two will questions will guide you on setting up the rolling merge. Here are two additional tips:

- I like to set the ?Ignore? period in the rolling merge to at least 3 to 7 days. This is my fine-grained incremental backup period. By never merging any layers in the past few days, I have access to hourly versions of my files should I discover I?ve done something stupid. And that happens more often than I care to admit.

- If you?ve got plenty of disk space, your merge and compact actions don't need to be performed every day. Schedule them to run once, maybe twice, a week. Schedule the merge to run before the compact.

Overall, it sounds like you?ve got things running pretty smoothly.

Help > QRecall Help > Guide: Automation > Actions > Capture > Adding special items to list

Like so many things about computers, it?s obvious ? once you know where to find it.

Alexandre Takacs wrote:How do I specify a hidden file or folder to be excluded from my backup selection. The UI allows be to specify finder-visible files, but what about the (more and more prevalent) hidden locations ?

I?ve got to start an FAQ.

When clicking on any + button to add items to an action:

hold down the Option key to pick invisible items

hold down the Shift key to pick items inside packages

Alternatively, you can use the Finder to expose the content of packages or use the Go to Folder command to navigate to invisible folders. Once there, drag anything into an action item list.

Alison wrote:First off, I am not very computer savvy, so need an easy-to-use back-up system that is very straightforward. Do I need much in-depth understanding to run QRecall?

QRecall is designed to be as intuative as possible. While there are a lot of powerful features, the basics (capturing backups and restoring) are pretty straight forward, some as simple as drag and drop.

I only want to back up certain folders (and implicitly sub-folders) such as Documents and Pictures. I don't want a full clone of the whole Mac drive. Am I able to do this with QRecall?

I would suggest starting with the Capture Assistent and have QRecall help you create a backup strategy. The assistent will ask you some questions and create a set of actions that will automate your backups and maintenace.

During this process, the assisent will ask you for the item (folder or volume) that you want to regularly capture. After the assisent is finished, open the Actions window and find the Capture action it created. Open this action and edit the items to capture. A capture action can capture any arbitrary set of items, from a handful of specific folders to multiple volumes. Edit this list, and that?s what QRecall will capture?nothing more, nothing less.

Also, how many machines can I use one license for? I have an iMac and my husband has a Macbook Pro.

To capture multiple machines to a single archive, you really need to buy a unique identity key for each machene. However, if you capture each machine to a different archive, you can reuse the same identity key. QRecall?s family-friendly license grants you the right to use the same identity key for every member of your household.

Kenneth Roe wrote:1. Is it possible to give users the ability to rename Capture Actions?

Short answer, yes. This has been on the to-do list for quite some time, it?s urgency has just never risen near enough to the top to actually get implemented. I?ll give it a nudge upwrards.

2. May I also second Norbert Karl's suggestion (May 2013) about allowing more flexibility in the delays imposed before a QR Action starts that is keyed to archive-volume connect?

This has already been added to QRecall version 2.0, along with new action events. In 2.0 you will be able to delay the start of any event action from between 1 and 999 minutes.

QRecall is a wonderful tool. Thanks for making it available?and for your work in supporting us out here.

You?re most welcome.

Charles,

In the QRecall application, choose Actions > Hold All Schedules > Until… and then pick a date far into the future (like, next year). When you return, choose Actions > Resume All Schedules and all actions that had been postponed will run.

Here?s a useful trick. If you have actions that have been held for days, and you don?t want to them to all run immediately when you choose Actions > Resume All Schedules, do this when you return: choose the actions you do not want to run immediately in the Actions window. Choose Action > Suspend Schedule. Now when you choose Actions > Resume All Schedules, only those actions that are still active will run. With the same group of actions still selected, choose Action > Suspend Schedule again to reactivate them. When an action is rescheduled, it finds the next run time in the future—ignoring any that it missed while it was suspended. Those action will then run at their next regularly scheduled time.

If you?re going to do this for most of your actions, you might find it easier to just suspend all of your actions when you leave and un-suspend them when you return.

Andre wrote:I have noticed a high cpu usage the last weeks.

This is a bug in OS X. It?s been around a while, and I really wish Apple would fix it.

It?s the result of a chain of events that starts when you choose the ?Exclude items excluded by Time Machine? capture option. When you select this option, QRecall makes a special call to OS X for each item it?s about to capture that asks if this is an item that Time Machine captures (or ignores). That call queries a database and returns an answer.

The problem is apparently in the database query. It appears to grab exclusive control of the database using the system preferences service. System preferences were recently (OS X 10.7) modified to use on-disk semaphore locks to ensure orderly access to property list files. This is a good thing, and it?s all managed by the cfprefsd process, AKA the Core Foundation Preferences Daemon. But it?s also where the bug is: sometime those on-disk semaphore locks get set and never reset. When this happens, any attempt to aquire the semaphore results in a ?spin lock? (a tight code loop that continuously checks the status of the lock, using 100% CPU) that goes nowhere and does nothing.

Back to QRecall. As QRecall checks every file to see if it needs to be captured up or not, it queries OS X, which then "spin locks" waiting to access the database. Instead of returning an answer in a fraction of a millisecond, it chews up 100+% CPU time for almost a second?thousands of times slower than it should, and at a huge cost in CPU resources. And, of course, this repeats for every single item QRecall is considering.

You might be able to avoid the problem by restarting your system. Most of the semaphore files will be reset when the system restarts, allowing cfprefsd to run efficiently again. If that doesn?t work, you can always turn off the ?Exclude items excluded by Time Machine? option.

Bruce Giles wrote:Any idea what's happening here?

When QRecall starts an archive action (capture, merge, verify, and so on) it launches a separate process that will perform that action. It then establishes a connection between itself and the new process in order to communicate the details of the action and any future cancel or pause messages. This communication is accomplished through a Mach kernel port.

When I first wrote QRecall, this worked flawlessly. Mach ports are, however, a limited resource; like file descriptors, there are only so many mach ports that can be used at any one time. As OS X has grown and the number of CPU cores, background threads, and processes continues to grow, it?s not that hard to run out of mach ports these days. When this happens, the process that's starting the action can?t open a commutations port with the process that's going to perform that action and ? well ? nothing happens, except for the log entry you?re seeing. The action process never gets its instructions, and eventually terminates without doing anything.

Mach ports can be registered under a name (via the mach port name server). Each port name must be unique, so the last eight characters of the QRecall port name is chosen at random. When the action process starts it picks a port name, registers it with the server, and then tells the starting process what that name is. The starting process then opens the named port and establishes its premanent communications link with the process.

I?m currently rewriting the inter-process communications logic that QRecall uses. I?m hoping to relay more on BSD pipes and less on Mach ports. I?m hoping this will make the communications between QRecall processes more predictable and robust.

Ralph Strauch wrote:The iPhoto Library is a package rather than a folder, and there doesn't seem to be any way to drill into it to exclude the plist file, so I guess I'll just leave that as is.

When adding any item to a list in QRecall (capture items, exclude items, ignore items, and so on), there are two ways of adding hidden items:

When clicking the + button to add an item, holding down the Option key will let you pick invisible items and holding down the Shift key will let you pick items inside packages.

Select the package in the Finder and use the contextual menu command Show Package Contents. Locate the item inside the package and drag it into the item list.

Otherwise, it sounds like you have things well in hand.

Ralph,

I don?t have much in the way of an explanation for why your layers disappeared on you, other than to suggest that you make sure you?re viewing the current owner and volume. If you have the Hide Unrelated Layers view option selected, and you inadvertently select an older owner or volume in the archive, QRecall will ?conveniently? hide all of the layers since that volume was last captured.

Reviewing the logs for your iMac, you?re consistently getting capture failures, but they?re rather minor. In fact, going back almost a month, the only consist problem you?ve encountered is capturing the tmSync.plist file in your iPhoto database folder. It would appear that some process is constantly replacing this file. When QRecall goes to capture it, it reads the directory, but by the time it gets around to reading the file, it?s been deleted and replaced. QRecall logs this and continues. It doesn?t, however, have any impact on the integrity of the archive.

(If you find this annoying, I might suggest that you add the tmSync.plist file to either the capture's ?exclude? or ?ignore changes? list.)

The MacBook Pro is another story. As you mentioned, it consistently fails (about once or twice a week) with I/O errors while reading or writing to the archive over the network. I/O errors are a deal breaker and terminate the capture, leaving the archive in a state that requires repair. Typically, the auto-repair is sufficient to get it back in service. But you also occasionally experience permission errors. I can?t tell if this is a file server issue, a virus scanner, or genuine permission problem. Permission or other access failures can leave the archive is a state that requires a full-blown repair or reindex to get things back into shape.

Which brings us to the repair actions. On the few occasions that you?ve perform a repair, the repair logged nothing out of the ordinary. This implies that the I/O and permission errors you?re encountering on the MBP are due to transient network transmission errors and are not a result of any damaged data in the archive or issues with the physical media.

If you?re ever concerned that your archive might have been compromised, run the Verify command/action.

One thing I will mention: I see that a Repair action was run from the MPB to an archive that (I assume) was accessed over the network. This is asking for trouble. If your network connection is causing spurious I/O errors, the Repair command will interpret that as damaged data and will delete that data from the archive. So if the I/O error was caused by the network, this will result in perfectly good archive data being erased?probably not what you want. If you must repair an archive remotely, start with the ?Reindex only? option. A reindex does not overwrite or attempt to repair the primary archive data file, so it won?t do any damage if the network causes problems.

Charles,

Welcome to QRecall, and welcome back from your vacation.

Each identity key defines an owner in an archive. All of the items you capture belong to the identity key you had installed at that time. QRecall uses identity keys to keep items from different computer systems from being comingled in a single archive.

A trial key is an owner that expires after two weeks. If you want to continue using QRecall, you can get a new trial key, but QRecall will treat all of your items as if they came from different computer system and capture them all over again. The ?penelty" of a trial identity key is that you have to start over every two weeks.

When you get and enter your permanent key, capture your items again. Open the archive and find all of the owners (the new one and the ones from your trial identity keys), select them all, and choose the Archive > Combine Items? command. QRecall understands that someday you'll need to change identity keys, or upgrade your hard drive, but that the items in that owner or volume are the same items you captured last week. The Combine Items? command transfers ownership of the earlier items to the new owner or volume, as if they?d been captured by that owner or from that volume all along. When it?s done, you'll have a single owner/volume with a single timeline for each of your items.

(You can?t combine the owners of two trial keys, but you can combine a permanent key with an earlier trial key.)

I hope that explains owners and trial keys. Post again if you have any other questions or problems.

Ralph Strauch wrote:I'm guessing that this is somehow related to Apple's new "Power Nap" feature, even though that option is not checked in my Energy preferences.

I would have to guess the same thing, except that you?ve got it turned off so it shouldn?t by waking up at all. Do you have a ?Start Up or Wake? option set in the capture action?

When I get your diagnostic report, I?ll file a bug with Apple.

I have Sophos anti-virus running and watching my email, and it occasionally quarantine's Windows viruses and asks me to remove them. If a backup runs before I've done that, a log entry will show up that the file could not be captured. See this morning's 9:43 backup, for example. Again, not a big issues, but just FYI.

That?s what I would expect to happen. Some anti-virus software will quarantine a suspect file, intercepting attempts to read it?s content at the filesystem level, to protect your system from its contents. That includes QRecall. QRecall will report that it couldn?t read the file, and continue on, and that?s exactly what you?re seeing the log.