Ralph Strauch wrote:Another advantage to mounting the archive as a volume would be that it would provide a column display, making it much easier to browse the archive.

Alternate views (column and icon) are the next addition to QRecall. I hope to have these in the beta by late June.

If you did that, would be mounted volume also allow browsing of layers, as the current display does?

No. The finder and OS will see it a standard volume. There are a number of ways of approaching this, and I haven't made any decisions yet. If I can manage it, it would be nice to see if I can "trick" the OS into treating the volume as if it were a Time Machine volume. If that were possible, you could use Time Machine's interface with a QRecall archive.

Otherwise, I'll either have to simulate a Time-Machine-like folder organization, where each layer is a top-level folder in the volume, or something else clever. There's still a lot of details to work out.

BTW, I've just found that backing up my new drive -- with the same name as the old one -- has created a new volume in the owners and volumes list.

Correct. Every "new" volume is treated as a unique entity by QRecall. "New" in this context means different. If you reformat or resize a volume, QRecall treats is a separate volume. You can delete the old volume from your archive (select the volume in the Owner and Volumes drawer and choose Delete Item). This will delete all of the archive information about the old volume. You might want to wait a few months before doing that. I also plan to add a command that will let you merge two volumes, making migration a little smoother.

Nicholas Sloan wrote:Is there any reason to suppose that things might be different on PPC/10.5.8 (which I am running now) from Intel/10.6.2 (which I expect to be running next week)?

I wouldn't expect so, unless there are differences in how these two operating systems behave that I'm not aware of.

I'm out of town this week. I'll review your diagnostic report when I return. In the mean time, please note any differences your see when you move to Snow Leopard, and feel free to submit another diagnostic report after trying it out on your new computer.

Ralph,

That's a brilliant idea!

I had been toying with the idea of writing a driver so that QRecall archives could appear as regular volume on the desktop. I had considered this idea somewhat of a novelty and hadn't really invested much time in it.

Exposing the archive as a volume, however, would allow Apple's migration assistant to use a QRecall backup as the source for a new install (assuming that one had captured the entire volume), which I think is a very significant advantage indeed.

I will definitely push this issue higher on the new feature list. It might require a little patience, though. Something like this is not likely to show up until 1.3 or 1.4.

Nicholas Sloan wrote:Hi, I am using and enjoying 1.2 beta, almost without event. Thank you.

That's always good to hear.

One small suggestion: in the Conditions section of the Action settings dialog, you can set an "Ignore Between" time span, but if you press + to add another condition and try to choose "Ignore Between" again, it is greyed out: so you cannot choose two separate time spans. Is this an arbitrary limitation, or would it be tricky to code to allow for multiple time spans here?

Neither, I just wanted to avoid confusion. It just so happens that I was thinking about this this other day because I have plans to add other conditions; conditions that would also make sense to add to the schedule more than once. I really just wanted to avoid the and/or anxiety that can come with some conditions. Thanks for pointing this out. I'll add it to the to-do list.

What brought this issue to the fore for me was finding my G5 awake in the mornings when it should have gone to sleep as normal after a SuperDuper backup scheduled for 3am. This seemed to be becoming a habit that only started around the time I updated QR to 1.2b, and I found that if I paused QR actions manually the evening before,it would not happen.

There could be a number of factors in play here. First, QRecall 1.2b1 uses a different (more modern) method of telling the operating system not to go to sleep while an action is running. The modern API behaves a little differently than the old one.

If you've added any power management options to your action schedules, this could cause your system to wake up. For example, one side-effect of asking the computer to go to sleep, restart, or shutdown after a series of QRecall actions have finished is that the OS will wake up the monitor to show you the dialog telling you that the computer is going to go to sleep. <Insert old joke about nurses waking up patients to give them their sleeping pills.>

There would be no QR progress windows showing, and I can't be certain that QR was implicated, but it looks like a probability.

The log should tell you what QRecall was doing, and when. You'll need to slide the details control all the way to the right to see insignificant events like power requests.

Jochen Schmidt wrote:A more than 5-fold speedup!!!
Is this actually expected?

Performance improvements for individuals is really hard to estimate. It all depends on your circumstances and configuration.

For most users, I expect mostest performance improvements. But in specific combinations of OS, archive size, RAM size, CPU type, and I/O configuration, the speed increase can be dramatic. I've created test cases here where the beta runs 40x faster than the previous version, capturing a test folder in 40 minutes where version 1.1.4 took over a day.

Ralph Strauch wrote:Are v1.2 archives compatible with earlier versions

No. Writing to an archive using version 1.2 will mean that earlier versions will not be able to use the archive.

For what it's worth, the core data format has not changed. If you use QRecall 1.2b1 to modify an archive and later want to revert back to using 1.1.x, reindexing the archive with 1.1 will make it usable again.

Charles Watts-Jones wrote:Having installed on a late 2007 iMac (iMac7,1) running 10.6.3 with 4 GB RAM, I turned off scheduled wake-up in Energy Saver and asked QRecall to wake and then sleep machine on completing its tasks. Worked flawlessly - great!

Excellent news!

I didn't however noticed a faster Verify (1:07:12 for a 237.6 GB archive) than I got with QRecall 1.1.4. Don't know if I should have expected this(?).

Verify was highly optimized a couple of versions back, and will be I/O bound for most users. In other words, QRecall can verify the archive as fast as it can be read from the drive. So until you get a faster hard drive/interface, verify won't get any faster.

The principle improvements in QRecall 1.2 are more efficient use of the quanta and packages indexes, expanded RAM usage (thanks to 64-bit addressing), and multi-processor load balancing. These changes improve capture performance the most, and impact compact and merge actions to a lessor degree.

Dawn to Dusk Software is pleased to announce the beginning of the QRecall 1.2 beta test program.

To get started, go to the QRecall Download page.

If you already have a permanent or trial identity key, you can continue to use that. If don't have a permanent key, or your trial key has expired, you can obtain a free Beta Identity Key that will be valid for the entire beta test period.

The theme of QRecall 1.2 is "performance and interface." The performance part involves a lot of under-the-hood changes and the bulk of that work is complete. Later beta versions will begin to incorporate a new interface, but I wanted to get the mission-critical code changes done first so they can get as much testing time as possible.

As always, feedback is welcome and encouraged.

Dawn to Dusk Software is pleased to announce the beginning of the QRecall 1.2 beta test program.

To get started, go to the QRecall Download page.

If you already have a permanent or trial identity key, you can continue to use that. If don't have a permanent key, or your trial key has expired, you can obtain a free Beta Identity Key that will be valid for the entire beta test period.

The theme of QRecall 1.2 is "performance and interface." The performance part involves a lot of under-the-hood changes and the bulk of that work is complete. Later beta versions will begin to incorporate a new interface, but I wanted to get the mission-critical code changes done first so they can get as much testing time as possible.

As always, feedback is welcome and encouraged.

David Cretney wrote:It seems random but the qrecall error is consistent. Is this a disk failing? I ran disk util repair permissions. Do I need to repair more?

I have been repairing the qrecall archive and I have another capture script that backs up the same imac disk to a different external drive and that one has not been producing these errors.

When encountering a data corruption error, here's my suggested course of action:

(1) Repair the volume. You did that, and that's good. Repairing a volume corrects volume and directory structure problems which can wreak havoc with other software (i.e. QRecall), and report all kinds of erroneous errors.

(2) Repair the archive using the default repair settings. Unlike most software, QRecall does not trust the reliability of your computer system or your disk drive. It confirms that every bit of information in an archive is correct and unaltered before using it. Repairing the archive will test every byte, discard anything that looks suspicious, and turn all of the vetted data back into a usable archive again.

Now, why is this happening? There are a number of situations where data can get damaged/lost:

Transient memory (RAM) errors can corrupt the data before it's transported to the disk drive.

The data can be corrupted while being transported to the drive's controller (via USB, Firewire, etc.)

The drive can fail to write the data correctly to the disk surface.

The data can degrade over time and become corrupted on the drive.

Perfectly good data on the drive can be overwritten with other perfectly good data because the volume's directory structure has become damaged, corrupting the archive.

The data could be misread by the drive.

The data read could be corrupted while being transferred (USB/Firewire) back to your computer.

The data read into memory could be spontaneously corrupted by a transient memory (RAM) error.

If the data is corrupted before it's written to the archive, or the data is changed/overwritten/corrupted on the disk drive surface, this creates a permanent data error in the archive. QRecall will complain that this particular bit of data is corrupt every time it attempts to read it. You can tell this is the problem by running a verify and looking at the log. Expand the "bad envelope whatever" message and look at the file position of the error. A permanent error will report the same file position every time.

An error that occurs while reading or transferring the data back to your system can cause transient errors. QRecall attempts to work around transient errors by re-reading any data that looks corrupt. If it's successful, QRecall will log a "Transient data error" in the log. If you find these, then your system is spontaneously damaging data as it's being read. The culprits are often the drive's controller, your USB/Firewire/etc. bus, or flaky RAM.

If the failure is transient but unrecoverable, then the problem is most likely the drive controller or data buffer. If you verify the archive multiple times and it fails in different locations, then it's a transient data problem that QRecall can't automatically recover from.

If you have one computer system that doesn't encounter any problems while a second computer system encounters data corruption errors—and you've verified that the volume's directory structure is OK—then the source of the problem is likely to be the second system's I/O or RAM.

Keep in mind that repairing a volume using Disk Utility or similar programs only corrects volume and directory structure problems. It does not verify the integrity or reliability of drive's data storage. This is largely due to the time and effort involved; a complete surface test of a modern drive would take days to perform and most modern drives have automatic data correction and sector sparing making testing somewhat redundant. But this doesn't mean that data is never lost, and is why QRecall is so untrusting of data storage devices.

David Cretney wrote:I'm trying to attach screenshots of a qrecall problem I have but I get this:

Apparently, the forum software is having directory permission issues. I'm looking into it.

Steven M. Alper wrote:The permissions entries are (I think):

2010-03-29 03:27:04.027 -0400 Details could not create lock file

2010-03-29 03:27:04.027 -0400 #debug# IO exception

2010-03-29 03:27:04.027 -0400 #debug# API: createFileAtPath:

2010-03-29 03:27:04.027 -0400 Details Path: /Volumes/WD Green (QR)/G5_whole.quanta/.lock

2010-03-29 03:27:04.028 -0400 #debug# OSErr: 22

BSD error 22 is "invalid argument"—which doesn't make any sense. This usually indicates that the volume structure is corrupt (which can cause the file system to spit out all kinds of nonsense error codes), or the operating system has simply become "confused." A restart will clear up the latter, and a disk repair will usually fix the former.

Steven M. Alper wrote:I think I've got the relevant log portion:

Nope. This log messages indicates a preallocation (disk full) error.

If you're encountering disk full errors, and your disk isn't full, you might want to read the threads "Leopard/Snow Leopard conflict when backing up to a network drive" and "Preallocation failed".

If this describes your situation, you can disable preallocation entirely by setting QRFilePreallocateDisable—see the post "Advanced QRecall settings"—or try to workaround the bug by changing the QRFilePreallocateBugWorkaroundRule setting.

James, I thank you again for your devotion to the software and support of your customers.

My pleasure.

Steven M. Alper wrote:I'm seeing repeated failures to multiple archives. As we've seen in the past it's probable that QRecall is not the cause, but I'm hoping we can eliminate it before I turn to other more expensive options.

The reported problems range from:

icon reference does not reference a data package

    which leads to:

The archive data is damaged or incomplete....

Steve,

This is a bug in QRecall. The repair command doesn't check the icon references of directories, which means that even after a repair the archive will fail verification. The good new is that this is entirely a cosmetic problem; the icon reference is simply so that QRecall can display the same folder image that the Finder does. It doesn't have any impact on the actual file data stored in the archive.

This has been fixed in 1.2b1, which I will get out real soon.

This is, of course, my big archive at about 237gb. I have run repairs on the archive which have always completed successfully, leaving a "Damaged" layer that does not seem to be delete-able.

You can't delete a damaged layer, but you can merge it with subsequent layers. A damaged layer is missing information which subsequent layers may have recaptured. Personally, I'd leave this layer alone until 1.2 is available. I've recently made significant enhancements and fixed a number of bugs related to the repair, merging, and recapturing of items with damaged layers.

Last night I had a verify with the "icon reference" error, followed by a successful capture. (Why doesn't a verify failure prevent further unattended actions on an archive?)

This is somewhat of semantic decision. The Verify opens the archive read-only, so (by definition) it can't modify the archive in any way, which would include changing it so that subsequent actions wouldn't run. And in your case it's a good thing, because the verify is reporting an error that repair won't fix, so you'd really be stuck if verify made your archive unusable.

In general, actions like capture, compact, and merge check what they need to check as they work. If anything is amiss, they will immediately stop using the archive to avoid corrupting the data in the archive. But if they don't encounter any problems, then more than likely they haven't made anything worse than it already was.

Previous to that I had a compact fail on the same archive with:

could not create lock file

     Path: /Volumes/XXX/xxx.quanta/lock

     Permissions: 0x0180

That's usually a permissions problem, but not always. Without the error code reported I can only guess.

The rest of the errors you report are typically caused by a corrupted volume or transient communication errors. I would suggest that you repair the volume using Disk Utility or your favorite drive repair software. If the repair reports any problems, then verify/repair the archive.

Ralph Strauch wrote:I just restored a folder that showed up in the Qrecall info window as 1.85gb in my archive and it turned into a 9gb folder on my hard drive. I guess that means that the size in the info window is the size of total data in the archive and the original data contained enough redundancy to cause it to balloon up like that. Is that right?

No, it's actually supposed to be the aggregate size of the original items. There are two potentially confounding factors:

The size of items in QRecall is their size in bytes, not their size on disk. If you have lots and lots of very tiny files, they may take up considerably more space. This is the discrepency between the "size on disk" and the actual number of bytes that you see in the Get Info window.

There have been long succession of bugs in QRecall that miscalculate the aggregate size of a folder. I thought I'd gotten them all, but I wouldn't at all be surprised if there were others lurking. I may bother you again when I get around to tackling this issue again.