Jan,

Here are two examples of your capture issues.

The capture started 2016-03-08 17:30 failed to (completely) capture the following file:

/Users/xyz/Library/Application Support/Google/Chrome/Default/History-journal

The problem given was "re-read past eof". This usually means that something was writing, or rewriting, that file at the moment it was being captured. (A similar error you might see is "file size changed".) If this file isn't particularly important, you can ignore this issue. If it is important, quit the Chrome browser (so the file isn't being modified) and recapture that directory.

The capture started on 2016-03-07 17:30 is a little different. It records the inability to capture a bunch of files (in the Trash), because of I/O error 2: "No such file or directory"

This is actually a bug, and has since been fixed. The capture is logging that it can't capture the file because the file has been deleted. Deleted files should just be recorded as deleted; they shouldn't cause errors to be logged.

If you still don't see any reason why an "encountered problems" message is appearing, send a diagnostic report (Help > Send Report) and we'll look into it.

Jan Sass wrote:Where do i find the capture log?

In the QRecall application, choose Window > Log.

Find the top-level log record of the capture in question and expand the groups to expose the details.

For an increased level of detail, move the Details slider (upper right corner of the window) to the right. To decrease the detail, slide it to the left.

Jan Sass wrote:I get this error on every newer backup; but even at highest level of detail i dont know what happened, what failed or what to do.

If you get this message, you should also have a "Capture issues" log record that contains a summary of all of the files/folders that encountered problems, grouped by directory, and what those problems were.

Capture problems range from the annoying to the serious.

At the annoying end of the scale, you may see "no file icon" or "could not obtain display name" issues. These are cosmetic; icon data is only used to display the items in the QRecall browser; it does not affect how the item is recalled.

Some issues have a variable amount of severity. The "file length changed during capture" issue is a common one. It means that the file's content was being added to while it was being captured. For some files (log files, in particular), this is a non-event because the portion of the file that was captured doesn't change. On the other hand, if this was a database file or disk image, that error could mean the captured copy of the file is incomplete or represents an an unfinished version of the file. When it detects this situation, QRecall automatically recaptures the file (up to three times) before logging this problem.

Then there are the serious problem: Insufficient permission to read file, I/O error, and so on. In these situations, it means the file was not successfully captured and may indicate a problem with the source volume or drive.

Two notes:

First, the beta version of QRecall logs a lot more information to its log than the release version. Because it of this, it's highly likely that you'll encounter a "file length changed during capture" issue on the QRecall.log file. This can be ignored (and will likely disappear in the release version).

QRecall currently logs a "file not found" error during capture as a serious problem. This will likely change, because it just means that a file was deleted after the capture process was started but before the file could actually be captured. This is really a non-event and should simply be treated as any other deleted file.

A QRecall archive is currently limited to 6 terabytes. Since QRecall's data de-duplication usually runs around 60%-80%, this would represent somewhere between 15 and 30 TB of captured file data. (That's a lot.)

This limit is for a variety of reasons, but mostly because extensive testing shows that archives greater than this size are only marginally usable. The amount of effort required to perform data de-duplication increases exponentially as the size of the archive grows. At the 5 TB size, it gets pretty sluggish. Note that there are new features in QRecall 2.0 that make capturing to large archives more manageable, but I've yet to have a customer tell me that they need a bigger archive. But if it's any comfort, there's no technical reason this can't be raised to 10 TB, and with some software work extended even beyond that.

There's no limit (per se) to the number of files, but there is a limit to the number of records in the archive database. Since each unique file requires at least two records (one record for the file metadata and one for its content data), there's a theoretical limit of about 200,000,000 unique files, or 400,000,000 duplicate files, in a single archive. But these would have to be atypically small files; you are much more likely to bump against the 6 TB archive limit first.

I've tried, and I can't reproduce this problem (at least on the three systems I tried it on).

Try to reproduce it, and then send a diagnostic report (in the QRecall app, choose Help > Send Report…). It may be a compatibility issue with an earlier version of OS X.

I've never heard of anyone complaining that QRecall and VMware don't play nicely—and I have quite a few VMware users.

First, make sure you're using QRecall 2.0 (currently in beta, but close to a final release). If your still running into problems, send a diagnostic report.

Alexandre Takacs wrote:Hi

Hello!

Probably a newbie question but I can't seem to find a way to allocate a fixed size - say 1Tb - to my archive a let QR manage it.

Not per se.

You can add a "free space" condition to your merge and compact actions that will only run when the free space on the archive's volume drops below a fixed threshold. The idea is that you let the archive grow as needed. When the volume starts to fill up, the merge and compact actions start to run and begin discarding older items until your free space is above the threshold again. That's actually one of the schemes the capture assistant will set up for you.

If you absolutely have to cap the size of an archive at 1TB, I'd suggest creating a 1TB partition for that archive. QRecall is smart enough to stop capturing when the free space on the volume drops to a critically low level.

There have been a few requests over the years for an "archive size is ..." condition, but it's still on the wish list. I'll give it a +1.

Also compact action is just unworkable with my (consumer grade, admittedly) NAS:

First, make sure you're getting the best performance you can. The compact action does a lot of I/O. If you have multiple computer systems, use the system with the fastest connection to the NAS (preferably wired Ethernet) to do the compact. You'll probably get an order or magnitude better performance over WiFi.

The compact action also performs the bulk of its work incrementally. That means you can stop the compact at any time and start it again the next day and it will, more or less, pick up where it left off.

To automate this, schedule the compact to start every evening (say 9:00 PM) on a computer you don't mind leaving on. In the compact action, add a "stop after X hours" or "stop at XX:XX" condition to stop the compact after it has run for, say, 9 hours. Let that run every night for a week, and you should get caught up.

The need/benefit of compacting an archive is sporadic and infrequent. Once compacted, you probably won't need to compact it again for a month or more (unless you capture and roll over a lot of data). The compact action is smart and won't do much, beyond basic maintenance, most days if the free space in the archive hasn't grown too much.

Bryan Derman has released rev D of his optical media backup script for QRecall 1.x.

Mostly minor bug fixes, but if you're using it you'll want to update.

See the release notes at http://www.derman.com/blogs/QRecall-Backup-Sets.

QRecall 2.0 will be a free upgrade for everyone.

All permanent identities keys will work with QRecall 2.0. So it doesn't matter if you buy a key now, or after 2.0 is released.

There are no plans to charge existing key holders for upgrades in the foreseeable future.

In the interest of full disclosure, there may be future features which require an additional identity key to take advantage of those features. But that would only apply to those new features, in the future...

Erik,

Sorry, but 2.0 is feature complete. It might make it into 2.1 or 2.2.

In the mean time, you might be able to take advantage of QRecall 2.0's ability to set capture preferences for individual items. If the files you want to exclude are relatively static, you could script the command line tool (qrecall) to set the "exclude" capture preference for a group of files matching a filename pattern.

Erik,

Thanks for the additional information.

cfprefsd (and yes, that stands for exactly what you think it does), can also simply get "stuck" updating a preferences file. Technically, it's a deadlock, and it is not always associated with a runaway cfprefsd process.

You'll notice that most of the preferences files have companions; the file com.qrecall.monitor.plist might have a file named com.qrecall.monitor.plist.CY0h1bN. These are temporary copies, waiting to be swapped out. Sometimes the semaphore managed by cfprefsd get orphaned, the preferences file it controls becomes deadlocked, and a stale temp file is left behind.

I suspect (hope) that cfrefsd will eventually break these stale semaphores and resume normal operations, but I've been too impatient to find out and usually just end up restarting my system to resolve it. I'll also, occasionally, delete these old temp copies (since cfprefsd never seems to clean up after itself).

Erik,

Thanks for taking the time to reproduce this.

I suspect that you're running into an issue with OS X. Starting around OS X 10.9, Apple switched to a controlled method for managing access to preference files. Access and updates are now controlled by a daemon named cfprefsd.

While generally a good idea, cfprefsd can sometimes get "stuck" updating a preferences file. What can happen is that you enter your identity key in the QRecall app, and then run another action (like capture) and that action complains that you don't have a valid identity key.

The problem is that the requested change to your preferences file made by the QRecall app (setting your identity key) is "stuck" and it could take a long time before the change gets made. Meanwhile, any action that tries to read your identity key gets the copy from the old value that is still on disk.

Eventually cfprefsd will update your preferences file and everything will work, but for a period of time in between it will mysteriously not.

Deleting all of your com.qrecall.* files in ~/Library/Preferences and/or restarting your system will also clear the logjam.

Erik Mueller-Harder wrote:I had misunderstood completely! I had thought that the beta keys were good for (approximately) the time specified, or until the end of the beta cycle, whichever came first.

Very close. A beta key is valid for the range of versions being tested, and those individual versions each expire on a particular date (listed in the release notes for that version). For example, a v2.0 beta key will work with all 2.0.x(x) beta versions. If you're running version 2.0.0(33) beta it will expire on April 1. If you're running version 2.0.0(19) beta, it has already expired (October 29, 2015).

I?m still running b32 on the machine I originally set up, and so the key is still working there. (I update each of my computers? software weekly, and it?s not that machine?s time yet.) Today?s experiments have been with other machines and with b33; hence the discrepancy.

Beta keys can be used with all beta versions with the same major and minor version number. Your current key will still be valid for all beta versions of QRecall between 2.0.0(29) beta and 2.0.0(33) beta. Earlier versions of the beta have already expired.

So if there's a problem on one machine, I suspect that you're using a different key (one issued for an ealier beta test), the version of QRecall is actually 2.0.0(26) or earlier, or the system date of your computer is wrong (in the future).

On another note, what is the best mechanism with which to submit minor beta feedback; e.g., UI glitches or ?Help? typos?

Whatever method you find easiest. You can send a diagnostic report, post something to the forum, or just fire off a note to support@qrecall.com.

I wouldn't worry too much about reporting anything that's wrong with the help, as it's is currently being rewritten for 2.0.

Erik,

Make sure you're using the current version of the beta and the beta identity key was issued for that beta.

A beta identity key is only valid for the matching beta version of the QRecall application it was issued for. For example, a beta key issued for the QRecall 1.2 beta will not work with the 1.1 or 2.0 betas, nor will it work with any released version of QRecall.

Your QRecall account page (https://secure.qrecall.com/account.jsp) lists the beta keys you were issued. Each beta key indicates which version it is valid for.