Ralph,

A last minute change in 2.0.0(37) introduced a race condition that could crash the browser when items are being loaded in the background. This bug has already been identified and fixed, but you'll have to wait until the next release, hopefully tomorrow.

You're not hitting any QRecall limits.

I suspect that you're hitting a (possible bug) in the sparse disk image that's corrupting the volume when you try to write a file that's larger than 1 TB.

I suggest writing the archive to regular volume (not a disk image). If that's not possible, try creating a regular (non-sparse) disk image and see if that makes any difference.

QRecall 2.0 requires OS X 10.8 or later. I'm sorry if that wasn't clear. I'm sure that was mentioned somewhere in the 400 pages of release notes...

Dr. D wrote:I am consistently getting a corrupt archive backing up to a fresh copy of a working archive, just around the 1 TB size.
As I was wondering if there is any limit in QRecall (or possibly in accessing the archive on a networked volume) ...

All filesystems have an upper file size limit. In most modern filesystems, the upper limit is so large as to be essentially unlimited, but older filesystems and file servers can be limit the size of your archives.

First, consider the filesystem (format) of the volume containing your archive. Ideally, it should be Mac OS Extended (also known as HFS+). This is the native filesystem for OS X and will handle archives of any size. If it isn't Mac OS Extended, consider reformatting it, if that's an option.

If the archive is stored on a remote volume then the combination of the server and server volume's filesystem will determine the maximum file size. You'll need to investigate both. For example, some Windows-compatible filesystems are limited to 4GB files, older AFP (Apple File Protocol) servers are limited to 2TB, and some Linux-based servers are limited to 1TB. Some even older filesystems, like FAT32, are limited to 2GB (not TB) files, but you would have hit that limit a long time ago.

From your other post, I suspect you're running QRecall 1.2.x. New code in QRecall 2.0 determines the maximum file size for a volume and uses that to limit the size of the archive. Unfortunately, that code isn't in QRecall 1.2.

If reformatting your volume isn't an option, or the limit is in your fileserver, consider splitting your archive. You might capture your System and Applications folders to one archive and your user folders to a second archive. Excluding items you don't need to capture (like log files) and turning on compression are other ways of reducing your archive's size.

Ralph,

The "problem closing archive" issue is probably unrelated. It appears that you left the archive open, and that computer later went to sleep, and while it was asleep some other process updated that archive, and when the first computer woke back up and tried to close the archive it found it in an unexpected state. Annoying, but by itself it doesn't mean anything bad happened.

The "invalid parameter" error is actually a red herring. The real problem happened a few seconds earlier when QRecall tried to duplicate the negative.index file. The copy failed with a "Timeout" error. This usually means the computer lost it's connection with the server, most like an interruption in WiFi.

That probably left the copy of the negative.index file half done, which explains the invalid negative hash map size error on the next attempt.

I suspect a simple repair (possibly even just a reindex) will fix the problem.

Ralph,

I wouldn't classify this as a "conflict". It's more "a trivial event that's being reported in the log as a problem because of a bug."

Basically all your log is reporting is that that a few files inside the invisible Genie9/Zoolz directory couldn't be captured because the files got deleted between the time that QRecall read the directory and the time it actually started to capture them.

That's actually a bug. There's no logical difference between a file being deleted before the directory is read and after, so this should not be logged as an error. The next version of QRecall will fix this.

Ralph,

What happens when you choose View > Show Toolbar?

Ralph Strauch wrote:I deleted that layer and reran the backup, and I'm back in business.

That's great news. This is also further confirmation that the error correction codes are, under specific circumstances that have yet to be fully identified, interfering with the repair process. (And the irony of error correction causing errors is not lost...)

Is it possible to exclude files within a package

There are several ways:

When adding excluded items to the archive's settings, hold down the Shift key when clicking the "+" button, and the dialog will allow you to choose items inside a package.

Use the Finder's Show Package Contents command, find the item, and drag it into the exclude list or use the Capture Preferences service to set the "Do Not Capture" preference. (Don't do this inside signed applications—it will alter the digital signature of the package and OS X will think it's been hacked).

Use the qrecall captureprefs command to set the "Do Not Capture" preference for one or more items.

and is it possible to exclude files by name regardless of where they occur?

That is a common request, and it on the short list of features to add in the next major version.

I miss being able to select and run an action from the same window. Not a big deal, but just a minor annoyance.

You should be able to do that now. Could you be more specific about what part of the interface you're using, what you're trying to do, and/or what's not working?

Ralph Strauch wrote:I shut down and restarted the iMac, interrupting the ongoing backup. The archive now appears to be unrepairable, ...

Ralph,

What leads you believe the archive is unrepairable?

QRecall has special mechanisms for dealing with exactly this kind of failure, because it's so common. The auto-repair feature should, under normal circumstances, automatically repair an archive that's experienced an interrupted capture or merge, and immediately restore it to its former state. The next action will then proceed as if nothing happened.

If, for some reason, auto-repair didn't fix archive, the repair action should most certainly reconstruct the archive, and preserve most (if not all) of your history. If the repair isn't working, then there's something else that's dreadfully wrong.

Also be aware that there are several issues regarding repair and data redundancy under investigation. If your archive has redundant data, try this:

1) In the Finder, select the archive, right/control+click, and choose "Show Package Contents"
2) Locate the repository.data file.
3) Find all of the repository_*.* companion files. They'll have names like 'repository_p8w8k16m2.0.anvin_reed_sol' and so on.
4) Trash all of the repisitory_*.* companion files, BUT DO NOT trash the 'repository.data' file.

Close the window and try to repair the archive again. Afterwards, send another diagnostic report so I can compare the results.

Jan,

Here are two examples of your capture issues.

The capture started 2016-03-08 17:30 failed to (completely) capture the following file:

/Users/xyz/Library/Application Support/Google/Chrome/Default/History-journal

The problem given was "re-read past eof". This usually means that something was writing, or rewriting, that file at the moment it was being captured. (A similar error you might see is "file size changed".) If this file isn't particularly important, you can ignore this issue. If it is important, quit the Chrome browser (so the file isn't being modified) and recapture that directory.

The capture started on 2016-03-07 17:30 is a little different. It records the inability to capture a bunch of files (in the Trash), because of I/O error 2: "No such file or directory"

This is actually a bug, and has since been fixed. The capture is logging that it can't capture the file because the file has been deleted. Deleted files should just be recorded as deleted; they shouldn't cause errors to be logged.

If you still don't see any reason why an "encountered problems" message is appearing, send a diagnostic report (Help > Send Report) and we'll look into it.

Jan Sass wrote:Where do i find the capture log?

In the QRecall application, choose Window > Log.

Find the top-level log record of the capture in question and expand the groups to expose the details.

For an increased level of detail, move the Details slider (upper right corner of the window) to the right. To decrease the detail, slide it to the left.

Jan Sass wrote:I get this error on every newer backup; but even at highest level of detail i dont know what happened, what failed or what to do.

If you get this message, you should also have a "Capture issues" log record that contains a summary of all of the files/folders that encountered problems, grouped by directory, and what those problems were.

Capture problems range from the annoying to the serious.

At the annoying end of the scale, you may see "no file icon" or "could not obtain display name" issues. These are cosmetic; icon data is only used to display the items in the QRecall browser; it does not affect how the item is recalled.

Some issues have a variable amount of severity. The "file length changed during capture" issue is a common one. It means that the file's content was being added to while it was being captured. For some files (log files, in particular), this is a non-event because the portion of the file that was captured doesn't change. On the other hand, if this was a database file or disk image, that error could mean the captured copy of the file is incomplete or represents an an unfinished version of the file. When it detects this situation, QRecall automatically recaptures the file (up to three times) before logging this problem.

Then there are the serious problem: Insufficient permission to read file, I/O error, and so on. In these situations, it means the file was not successfully captured and may indicate a problem with the source volume or drive.

Two notes:

First, the beta version of QRecall logs a lot more information to its log than the release version. Because it of this, it's highly likely that you'll encounter a "file length changed during capture" issue on the QRecall.log file. This can be ignored (and will likely disappear in the release version).

QRecall currently logs a "file not found" error during capture as a serious problem. This will likely change, because it just means that a file was deleted after the capture process was started but before the file could actually be captured. This is really a non-event and should simply be treated as any other deleted file.

A QRecall archive is currently limited to 6 terabytes. Since QRecall's data de-duplication usually runs around 60%-80%, this would represent somewhere between 15 and 30 TB of captured file data. (That's a lot.)

This limit is for a variety of reasons, but mostly because extensive testing shows that archives greater than this size are only marginally usable. The amount of effort required to perform data de-duplication increases exponentially as the size of the archive grows. At the 5 TB size, it gets pretty sluggish. Note that there are new features in QRecall 2.0 that make capturing to large archives more manageable, but I've yet to have a customer tell me that they need a bigger archive. But if it's any comfort, there's no technical reason this can't be raised to 10 TB, and with some software work extended even beyond that.

There's no limit (per se) to the number of files, but there is a limit to the number of records in the archive database. Since each unique file requires at least two records (one record for the file metadata and one for its content data), there's a theoretical limit of about 200,000,000 unique files, or 400,000,000 duplicate files, in a single archive. But these would have to be atypically small files; you are much more likely to bump against the 6 TB archive limit first.

I've tried, and I can't reproduce this problem (at least on the three systems I tried it on).

Try to reproduce it, and then send a diagnostic report (in the QRecall app, choose Help > Send Report…). It may be a compatibility issue with an earlier version of OS X.