QRecall 1.0.0(49) changes how the background scheduler process (daemon) is installed and run.

In prior versions, the scheduler was starting using a cron job. Starting with version 1.0.0(49), the scheduler is installed as a launchd daemon (in /Library/LaunchDaemons).

This should resolve a number of problem that people have been experiencing with Mac OS 10.4.10 and 10.5. Specifically, the scheduler process was being unceremoniously killed by the operating system when the user logs out or restarts. This was causing a raft of problems, ranging from messages about broken communications pipes, not running actions when they were scheduled to run, and leaving actions stuck indefinitely "waiting."

As a launchd daemon, the scheduler should now be running all of the time and doesn't have to be periodically restarted. This means less overhead, more predictable execution of scheduled actions, and QRecall won't be affected by customized installations of cron.

The first time you launch QRecall and it detects saved actions, you save an action, or you try to schedule or run an action interactively, it will prompt you to install the scheduler. You may be required to provide the name and password of an administrative user.

So you might be asking why the scheduler hasn't always been a launchd daemon? The reason is that this change breaks a big promise in QRecall: Namely, that all QRecall features are available to non-administative users, making QRecall usable in a lab or shared computer environment. I felt very strongly about this promise, but fighting Apple and their ever-changing treatment of background processes, cron, security, and kernel namespaces has made this impossible.

The interface for installing (and uninstalling) the scheduler will get some refinement in upcoming releases. In the mean time, please provide any feedback you have about the new scheduler installation process or any problems with the scheduler.

I wonder if this explains the problem I've been having on my sever.

Very likely.

Version 1.0.0(49) should resolve this problem. It's finished and available on the alpha release feed. It's been undergoing testing here for the past two days. If all goes well, I'll release it to the beta feed tomorrow.

Stephen Murphy wrote:Right now one big difference for me is that Time Machine can only backup to an external disk -- Apple pulled the Airport Disk functionality right before release.

That surprised me too. I have gotten no clear answer why Apple pulled the network volume support for Time Machine at the last minute. I can only guess that it was a performance issue.

On my laptop, I don't want to have to have an external drive plugged in all the time, and Qrecall works just fine with my Airport Disk.

I agree. If I couldn't back up my two laptops via Airport, I probably wouldn't be using QRecall either.

Ralph Strauch wrote:Time Machine appears to have a "prettier" and more transparent UI. The ability to go back and see old versions with QuickLook looks nice.

Having the recall GUI built into the operating system is a definite plus for Time Machine. Sadly, not an option available to third-party developers.

I think Time Machine backs up changed files rather than changed sectors, so it's archive will grow much more quickly than QRecall's. QRecall, OTOH, has a "geekier" UI requiring a bit more computer savy, but will use much less space for its archive or be able to save more layers in the same space.

Also correct. Time Machine, like most incremental backup solutions, copies entire files whenever they change. So small changes to large files will require considerably more storage.

Is a good summary of the differences, or is there anything important that I'm missing here?

The biggest differences today are efficiency (which you've already commented on) and control. Time Machine has one, fixed, backup schedule. QRecall gives you almost unlimited control on what gets captured, when, and how long it's saved.

My long term goal is to capitalize on QRecall's flexibility and control, so that it will eventually become "Time Machine Pro."

The space difference is a biggie, and this week when people are installing Leopard and exploring Time Machine would be a good time to promote that heavily. In the MacIntouch reader report on Leopard, http://www.macintouch.com/readerreports/leopard/ a number of people are expressing concern about Time Machine's archive growing rapidly because of small changes to large files. They'd likely be good candidates for QRecall right now. I'll post something about it if you'd like me to, but it might be better if you did it yourself.

I find that this sort of advice is better from actual users. When I post, it sounds self-serving -- which, of course, it is.

Ralph Strauch wrote:Why are two different keys needed?

When you reuse a single key on two systems, QRecall treats both computers as the same source. So from QRecall's perspective, you are capturing two volumes on the same computer rather than the single volume of two different computers.

The second reason is economic. Using QRecall's ability to combine duplicate data (like the OS) from two diferent systems is a significant advantage that QRecall has over almost all other backup systems available today. Using two identity keys let's me get paid for that work.

Related question? should it be possible to have a firewire drive mounted on both computers at the same time?

Not directly, no. A directly connected Firewire drive can only be used by a single computer at a time.

To share a volume with two or more systems you need to share it (there has to be some intermediate agent that arbitrates requests between the multiple systems). You could, for example, connect the drive to one computer, turn on Personal File Sharing, then mount the same volume on the second computer as a network volume. The system directly connected to the drive will arbitrate acces to the drive.

Frederic Thomas wrote:Did you get the logs? I'm just asking since I did not get a reply from you and it was a 5.x MBytes email...

Yes, I got the logs. I just haven't had a chance to look them over yet.

Also I am trying capturing something from OS X now, the volume that was giving us trouble.

Keep us posted...

Frederic Thomas wrote:My problem is (a) at some point I really need a working system and (b) disk space becomes a problem running multiple systems in parallel.

So I had to use the backup disk space for a working backup with Retrospect and had to delete the old archive: not easy to move around 250GB...

Not a problem. I understand completely.

Kept all the log files, sending them now.

Very much appreciated.

I am not sure I understood what test you'd like me to perform: OS X Server stuff on a OS X server only archive ? (even a local one?)

If the working theory is correct, that issues that occur when running OS X server are causing the archive corruption, then it should occur if captured to its own archive. So the test would be to set up the server to capture to an archive and see if the problem reoccurs.

But if you don't have the time or space to do that, don't worry about it. I still need to fix the missing icon resource problem and do some more investigation.

I've found out what the problem is.

A recent change to OS X now kills (SIGKILL -- essentially a force quit) any process belonging to the user when they log out. Worse, it kills any background process that tries to start up again.

This change makes it impossible for the scheduler to run while logged out.

I hope to have a solution soon.

I forgot to mention in the previous post, please send me those log files. I'd like to examine them for any additional clues about the missing icon problems.

The comment I made about reinstalling the OS was based entirely on my experience with OS X, not OS X Server. The call that's caused your capture problem has never failed on any regular system I've ever encountered.

OS X Server appears to "think different" when it comes to the launch services database.

I'd very much like to get more feedback from users that are using QRecall to capture multiple systems to a single archive. For the time being, would it be a terrible inconvenient to limit your multi-system use to just OS X client systems? The more I think about this problem the more I suspect an incompatibly with OS X Server which could be tested/confirmed by capturing to its own archive. This would be a smaller problem to examine and wouldn't interfere with the backups of your other systems.

First let me say that I'm extremely perplexed and upset about this situation. I can't come up with any scenarios to explain why an archive that passes a repair could contain a package with an invalid type (which is what the logs are reporting).

Given the other evidence, I have to assume that it's somehow related to the missing icon data that was reported in your earlier report. I'll try to simulate the same errors and see if I can reproduce the problem here.

You'll obviously have to repair the archive again before it can be used. After the repair, it would be useful to perform a verify. That would tell a lot about the integrity of the archive before anything else is done to it.

When diagnosing problems like this, the next step would be to produce a dump of the archive and analyze that. However, dumps of large archives can, themselves, be very large. Trying to send me a copy of the dump could be impractical.

It won't tell us what's going wrong, but you can probably reclaim the use of the archive by deleting all of the recent layers added by the systems that were encountering the capture problems. Performing a compact after the delete should clear out all of the detritus and allow you to capture your original system to the archive once again.

QRecall has not been tested on Mac OS X Server. There may be lurking compatibility issues which are at the root of this problem.

Then I need to theorize a reason why your log records would stop when you log out. Are you using FileVault by any chance?

The biggest performance gain by logging out, in my experience, is that it forces all of the user processes to quit and restart. This releases all of the stale file references and memory fragmentation that accumulate over time and starts over fresh.

Whether that little bit of extra performance compensates for the time spent logging out and back in is open to debate. But if it helps you, there's certainly nothing wrong with doing that.

Fred,

Thanks for the detailed problem report.

The "Could not obtain icon" and "No file icon" messages are exceptional, but not problematic. These messages mean that QRecall asked the operating system for the icon to display for these particular items and the operating system said it doesn't have one or couldn't find one. Odd, but nothing serious. It might suggest that your launch services database needs resetting. This can be done via the command-line and I'm sure there are free-ware utilities that will do this also.

Now, the shocking problem is the "error getting generic folder icon" failure. This simply should never, ever, happen and why it's coded as an exception in QRecall. This is thrown when QRecall asks Mac OS X for the generic document and folder icons for use document/folders that don't have a custom icon.

These should be static resources in the operating system, and the OS should always return them. Something is amiss here.

Clearly QRecall needs to deal with this situation gracefully. I'll fix QRecall to handle this situation. In the mean time, I'd consider reinstalling the OS on that computer.

Charles,

I had a chance to review your log files, and I have a few questions.

What the log files are showing me is that everything hums along nicely until about 22:30 in the evening, then the log entries abruptly stop. About 08:30 the next morning processes start up again. But there is no log entries indicating that the old processes were ever shutdown between 22:30 and 08:30.

Are you shutting down your computer at night, or is there some possibility that it simply being turned off or crashed (without going through its complete shutdown procedure)?

What's happening is this: The scheduler creates a semi-permanent communications port that it uses to talk to other processes. When the system boots up in the morning, the port is still there but there's no scheduler process connected to it. QRecall recovers by deleting the port and creating a new one, but not before you get those "Problem connecting to scheduler" warnings.

Hello Charles,

I'd be interested in seeing a copy of your entire log file, especially the period of time that you upgraded.

To be compatible with Leopard, b48 uses a different inter-process communications method. It sounds like it's eventually connecting (since its reporting the correct next action times and I assume that actions are running when they should). But it looks like some trash is getting left behind, possibly between reboots, that's not getting cleaned up.

Pessimists are never disappointed, and often pleasantly surprised.

Today we're pleasantly surprised.

QRecall version 1.0.0(48) is Leopard (Mac OS X 10.5) ready. I was seriously concerned that some portions of QRecall would require major changes in order to get it to work under Leopard. As it turns out, a couple of elegant and straightforward solutions were found.

Choose QRecall > Check for updates... to get the latest version.

The details of the changes can't be discussed until Leopard is officially released.

Caveat: While we've encountered no significant issues with QRecall running under Leopard, testing has been extremely limited and confined to the last Leopard seed released to developers. It has not yet been tested on the final release of Leopard. I'd like to remind everyone that QRecall is still beta software (even more so under Leopard); Please use it cautiously and report any problems you encounter.