Adrian,

Several people seem to be having trouble with the monitor process hanging up (and then hanging up everything that depends on it).

In the future, try to use QRecall's Send Report... command for reporting problems. In QRecall 2.0, Send Report samples the QRecall processes so I can tell what they were doing (or not doing) at the time.

And what version of OS X are you running?

If you can run the QRecall application, send another diagnostic report. (The report will sample the monitor process, if it's running, to tell me what's going on.)

Next, try using the Activity Monitor to kill the QRecall Monitor process and see if it relaunches. That might fix it right there.

If not, try restarting and see if that helps.

Paul Mann wrote:I sent you a log via Recall so hopefully you can see what is going on.

Good. (I was sweating the Send Report... feature on 10.11, so that's a relief.)

Edit: I forgot that I do not seem to have the follow options in my services menu either ;

My bad. I forgot to mention that OS X disables new services by default. You'll need to go into the services preference (either from the Finder > Services or via System Preferences) and turn them on.

The good news:

I was able to fiddle around with QRecall running under OS X 10.11(beta3), fixed a few issues, and it seems to be working. I was able to create an archive, run a capture, verify the archive, schedule an action, and so on.

The bad news:

El Capitan complains about the executable format of the embedded Sparkle framework in 2.0.0b6. (This is the tool that allows QRecall to auto-update itself.) I upgraded to the latest version of Sparkle and that fixed those warnings. But now there's a new problem: when QRecall tries to update itself, it crashes. This only seems to occur on El Capitan. One my 10.10 systems, QRecall auto-updated itself just fine.

So here's the deal...

If you want to play around with QRecall on 10.11, you can click this link to download version 2.0.0b7. Open the disk image and launch the Install QRecall app.

When the next version of QRecall becomes available, it will most likely crash. To work around that, manually download the next version and run the install program again. You'll have to repeat that process until the Sparkle issues are sorted out.

Paul Mann wrote:This is what I mean, there are 3 databases in 3 different locations on remote shares and I cannot see where any of them are?

As you pointed out, the Reveal in Finder command won't mount a remote volume just to show you where the archive is. I could change that behavior.

Another enhancement would be to add a help tip to the status window that would display the archive's original path.

You could also give your archives different names.

Paul Mann wrote:I did wonder though, are you planning on updating the look to be a little more Yosemite native, or is the plan to leave that part alone?

I plan to do some clean up of the UI, but the focus of this release is data integrity, reliability, stability, compatibility, and security.

Its looks a little dated albeit thats secondary to the data of course!

At this point, I feel it's more important to have a reliable system than a pretty one, although both would be nice. Also, my graphics design guy is currently working on other projects, so I probably won't be able to get any updates to the icons and such until early this fall, and that's squarely in the next release too.

Edit: Perhaps a harsh comment, but not meant to be, as after using a little more there are subtle nice changes.

You're a beta tester. Harsh comments are what I (don't) pay you for.

As a new user I struggle with not seeing for example 'where' the archive is located that is being actioned, in any of the dialogs showing actions being done.

For me it says capture db name to and nothing more.

That isn't a feature. The problem in El Capitan is that QRecall can't seem to create URLs or bookmarks or whatever to the repository. All of the errors you posted earlier are like "archive: (null)". The UI is supposed to show you the name of the archive, but it's broken internally and that's why it's blank.

Paul Mann wrote:Hopefully you are planning on supporting El Capitan in this v2 release, or not?

I plan for QRecall 2.0 support El Capitan. In fact, the reason I've been sweating this beta to get QRecall out there and tested with El Capitan before its release.

And being so focused on getting the beta working has meant I've had no time to test on El Capitan at all. But that's about to change. I'm typing this on a freshly installed 10.11 beta 3 system and plan to fire up the developer tools later this morning.

So for all of you brave El Capitan testers out there, sit tight. I'm sure we'll have something working soon.

Caveat: I'm already aware that there are new security features of El Capitan that will prevent QRecall from restoring a bootable El Capitan system. I'm investigating those and hope to have a solution soon.

Dawn to Dusk Software is pleased to announce the beginning of the QRecall 2.0 beta test program.

This rates a "finally!"

To get started, go to the QRecall Download page.

Please, please, please, read the release notes. This is a major upgrade to QRecall and will irreversibly modify your archives.

If you already have a permanent or trial identity key, you can continue to use that. If don't have a permanent key, or your trial key has expired, you can request a free Beta Identity Key on the download page that will be valid for the entire beta test period.

The theme of QRecall 2.0 is "moving forward." QRecall 2.0 jettisons many legacy OS X technologies, like the Carbon and File Services APIs. It has been rewritten using the latest compiler and language features, and incorporates many of OS X's advanced technologies.

This has paved the way for new features such as encryption, error detection and correction, native notifications, better security, per-item capture preferences, a command line tool, and much more.

Please note that, as of this posting, QRecall 2.0 has not been tested on OS X 10.11 (El Capitan) and has had very limited testing on OS X 10.7 and 10.8.

QRecall 2.0 is largely feature complete. A few minor features are still planned, but this beta test period is primarily to discover and iron out any problems with these new technologies.

As always, feedback is welcome and encouraged.

Lowell,

Thanks for your interest in QRecall. Let me see if I can answer your questions.

Lowell wrote:I want to add a folder hierarchy to an archive, but I don't want to include a couple of large subfolders. I don't see a way of doing this directly. As soon as I add the top level folder, QRecall immediately starts the capture process. I realize that I could wait for the capture to finish, then delete the undesired subfolders, but I would rather not wait for hundreds of GB in these subfolders to be backed up, just to immediately delete them. Is there a better approach?

Method #1: Using the current version of QRecall, create an action to capture the top level folder. In that action, add the folders you want to exclude to the Excluded Items list. Save the action and run it whenever you want to capture that top-level folder, but not capture those specific sub-folders.

Method #2: Wait for the QRecall 2.0, which will exclude items on a per-item basis. You can either set items that you never want captured in your archive's preferences, or set the capture preferences of any item right in the Finder. Once set, these exclusions apply to all captures, no matter how they are made (interactively or by actions).

Another unrelated question - when entering administrator access in QRecall, where is the password stored, and is it in clear text?

QRecall simply asks OS X for elevated permissions; QRecall never has access to your password.

When any application asks for elevated permissions, OS X puts up the "enter your admin name and password" dialog. It then verifies your identity and, if it's correct, temporarily grants the application the permissions it was asking for. The application never sees, nor has access to, the information you entered in the security dialog.

In QRecall's case, it originally asks for elevated privileges so it can install its helper tool as root. Once installed, the helper tool implicitly runs as root (so it can capture files your user account wouldn't normally have access to, like the operating system). This is a permanent condition, so QRecall doesn't have to ask again in the future. (Note that this mechanism also changes in QRecall 2.0, which uses a privileged helper service managed by launchd, which is more secure.)

Our database and email server experienced a slow and ugly death starting around June 22, 2015. The defective hardware was replaced, and the server restored (using QRecall, of course) on June 24, but all transactions between those two dates were lost.

If you uploaded a diagnostic report, sent an email message to support or sales, or created a QRecall account during these three days, please either send it again or log in and verify your account.

If you purchased an Identity Key during this time, you will be contacted directly to have the key replaced. (We have a record of all sales, which is handled by a different server.)

We apologize deeply for this inconvenience.

Matt Washchuk wrote:Do you have any suggestions on how I can keep bandwidth utilization to a minimum?

Not really, because QRecall already tries to minimize the amount of data read and written to the archive as much as possible.

QRecall performs block-level de-duplication. When it captures a file, each block of data of the file is first read. QRecall generates a hashcode from the data of that block. It then looks in the quanta index of the archive to see if a data block with that hashcode has already been added to the archive. If not, the data is unique and is added (written) to the archive. If it has, the matching block in the archive is read and compared to the block in the file to make sure they are identical (which it most likely is). This repeats with the next block until the file has been captured. (There are a few exceptions to this, but that's what QRecall is doing 99% of the time.)

So it's about as efficient as it can be. In general, capturing a 500MB file will cause either 500MB of data to be read from the archive, written to the archive, or some mixture of the two?but about 500MB is going to fly across the wire.

There is one optimization that will reduce this. It occurs when you're re-capturing a large file with very little changes (like a log file or a disk image); in that case, QRecall may skip reading the data blocks in the archive for some of the blocks in the file, significantly reducing the amount of data that has be transferred.

I guess I already assume it would be best to run the verify/merge/compact actions directly from the file server rather than the local machines, ...

That's a very good guess, and absolutely correct.

Bruce Giles wrote:So I saw you hint in another message that another beta cycle might be coming up soon. Care to comment any further on that yet?

QRecall 2.0 has been a study in excitement and frustration. There are a lot of new features, and most are working brilliantly. I've had torture tests running on a dozen separate archives for months now without any major issues.

But there are also several show-stoppers that I've yet to work through, mostly because I'm still mired in under-the-hood changes to modernize the code and bring it up-to-date with Apple's latest APIs and compilers.

It seems that almost every week I think "next week I'll get the beta out," and then I discover something that puts it off another week (or two). I've stopped trying to guess at when it will be ready for its debut, but I still feel like it's close. The point is, it's coming, but not before I'm satisfied that it won't lose anyone's data.

Ralph Strauch wrote:This morning, today's (network) backup of the MBP failed showing "size of negative hash map not power of 2" and " A network or disk error was encountered."

That's a very unusual error. If I had to guess, I'd say that something went wrong as the archive was being closed. The negative map is only resized a few times during the lifetime of the archive, and its length is always a power of 2. So for something to change that is unusual.

(I've learned that all software has bugs, even filesystem software. Something things just don't happen the way they should.)

Those errors also showed up when I tried to backup the iMac (directly mounted) and when I try to backup the MBP with the drive mounted there. I get the same errors when I attempt to do a verify.

That's correct. Every time the archive is opened, QRecall performs some basic sanity checks. QRecall will refuse to use the archive until this problem is corrected.

Bruce Giles wrote:QRecall immediately attempts to run both the both the full backup and the home folder backup. I presume this is because some of the scheduled events occurred during the period that the Mac was off, and now QRecall is trying to catch up for the missed events.

Correct. If the computer is shutdown (technically, if the QRecall scheduler isn't running), the scheduler will immediately run actions that were scheduled to run while it was shutdown.

If the event has already passed, I would prefer that QRecall just forget about it and wait for the next scheduled event. I could swear it used to work that way. Did I mis-remember, or did I do something wrong when setting up my scheduled tasks?

Bruce, you aren't mis-remembering. As one of QRecall's oldest customers, you probably remember when QRecall didn't run actions while the computer was asleep or shutdown. This generated a lot of complaints and the scheduler was modified to find, and immediately run, actions who's next scheduled time was now in the past.

I thought that maybe "ignore while user is logged out" ... would do the trick, but it didn't.

This depends on whether you're running the scheduler in the background or not. In QRecall > Preferences > Scheduler you'll find an option to "Start and Run actions while logged out". This option controls whether the QRecall scheduler process runs as an agent (only while you're logged in) or as a daemon (installed in the system and run all the time). If this option is off, the scheduler is running as a user agent, which means it start up when you log in and shuts down again when you log out.

If it's running as a user agent, the "ignore while user is logged out" option is worthless; it can't be tested because the action can never be started while you're logged out.

Solution #1: If your system doesn't auto log into your account, you could set the scheduler to run as a daemon and add the "Ignore while user is logged out" option to your capture actions. The computer will start up, the scheduler will start, the actions will try to run, but won't because you're not logged in yet.

I thought that maybe "ignore if no archive" ... would do the trick, but it didn't.

That condition will only skip the action if the volume containing the archive isn't mounted/online at the time that action wants to run.

Solution #2: Add the "ignore if no archive" to your schedule. When you turn on your Mac, don't turn on your external drive. The scheduler will try to run the tardy actions, but see that there's no external drive and skip them. Once you're logged in, turn on your external drive and captures will begin normally.

When the backups start running just after login, I usually cancel them from the monitor window. However, if I do that, and then reboot later, the backups attempt to run once again after login, even if another scheduled event hasn't yet occurred. Any ideas?

That's hard to say. If four hours have passed, or you just logged out and four hours have past, I would expect the capture with the interval schedule to run immediately when you log back in. If the daily action is running immediately after being canceled, then something might be wrong. Start by sending a diagnostic report.

Finally, I'm considering adding an "ignore if asleep or shutdown" condition to actions in QRecall 2.0 so you can simulate the original behavior.

Ralph Strauch wrote:What do you think about adding qrecall options to mount an archive volume before beginning a capture action and unmount it after that action as a defense?

I don't have to add an option, because QRecall already does that.

If the volume containing an action's archive is on a local, unmounted, drive then QRecall will first request that the volume be mounted. Similarly, if the archive is contained on a networked volume, QRecall will attempt to connect the networked volume before the action runs.

QRecall also knows its kindergarten rules: if you get it out, put it back when you're done. If QRecall caused a volume to mount or a networked volume to connect, it will request that the volume be unmounted/disconnected once the last action using it has completed.

Mounting and unmounting of an attached disk can be done with Disk Utility, but I don't see any way to do that with a disk attached to a networked computer. I'd like that option because I leave my archive attached to one computer but also back up a second one over the network.

That is beyond QRecall's reach. There's no protocol for requesting a volume be mounted/unmounted on a remote computer.

If a ransomware attack is a real concern, the best defense would be to use a set of rotating backups; back up to two (or more) drives that get swapped out about once a week or so. Since you never had both volumes connected at the same time, there's very little chance that malicious software could corrupt both before you detected it. QRecall has scheduling options that will let you ignore the actions for the archive(s) that isn't online right now, so you don't have a one set of actions that's always failing.