James Bucanek wrote:My concern is that there's a lot of stuff in /Library and /var/db that you might want and just migrating /Users and /Groups is not going to include.

Ah, now I understand. For this server, I don't think it's going to be a problem. A few years ago, we used it for a lot of things -- it was our network firewall, it was a license server, it was our DHCP server, and our DNS server, etc. But over time we moved all those functions over to other hardware. The only thing the Xserve does any more is act as a file server and run OpenDirectory. The users are all running Windows machines, and they don't even know it's an Apple server running Mac OS. They store only data files on it. We no longer run any third-party apps on it other than QRecall, or even any Apple apps other than OS utilities. So as long as we can get all the data transferred to the mini, I don't think there's anything else we'll need from the old server. But like you said, if that turns out to be wrong, there's always Plan B.

-- Bruce

I'm not going to transfer over ANY of the old operating system or server applications. In fact, I don't think that would work, even if I wanted to. The Mac mini is much newer than the Xserve, and while I might be able to use QRecall to copy Snow Leopard Server onto it, I doubt it would boot. The mini already has the latest High Sierra OS and the latest Server app, and I intend to stick with those.

What I do want to transfer is all the user (/Users) and group (/Groups) folders, along with the OpenDirectory database. (I'm somewhat concerned that the OpenDirectory database format may have changed enough I can't import the old database into OpenDirectory in the current Server app, but that's my problem.) If I can get all the User and Group folders transferred, that will accomplish most of what I need to do.

So you're saying that if I install the latest QRecall on the mini, and then connect the external drive that contains the backup created by QRecall 1.2.3.8, I should be able to restore those files and folders to the mini?

Also, will it make any difference if the server drives are APFS?

-- Bruce

James,

I have an old Xserve running Snow Leopard Server 10.6.8. It's been successfully backed up for years with QRecall, and it's currently running QRecall version 1.2.3.8. I'm planning to replace the Xserve with a newer Mac mini, running the latest version of High Sierra. Somehow, I need to transfer the top-level /Groups and /Users folders from the Xserve to the mini, along with the OpenDirectory database (I have the Server app on the mini.) I do not need to transfer system files, applications, or anything else. The backup archive is on an external drive.

The OpenDirectory database is small, and I have an export of it that I'm hoping will load into the Server app on the mini. I can transfer the database on a USB stick. The tricky part, I think, will be moving the Groups and Users folders, which have about 750 GB of data. Ideally, all files and folders need to retain their group and user IDs and protections, so that I don't have to reset all that after I get them moved.

Can I do that with QRecall? If so, do you have any recommendations? For example, should I load the old version of QRecall on the mini (if that will even work) and use that to restore the files to the mini? Or can I put the latest QRecall on the mini and then use that to restore the old archive? Or is there a better way?

-- Bruce

I see you've implemented it now, and it's working. Thanks!

-- Bruce

I have an action that backs up my home folder at 4 hour intervals. As long as the computer is running, everything works the way I want it to. If I shut down the computer, and then the next backup interval occurs while the computer is off, nothing happens, because I have not enabled the option to start up my computer. This also works exactly the way I want. However, the next time I start my computer after that, then the interval action will run immediately, and that is NOT what I want. Is there a way to tell QRecall to ignore actions while powered off, and just resume at the next scheduled interval after powerup? I looked at the various condition options, but I didn't see one that would do that.

-- Bruce

James Bucanek wrote:This is a little on the experimental side, so I'm looking for feedback.

QRecall no longer reports the progress of individual items inside a directory if that directory is either opaque (package, application, bundle, etc.) or if it is either invisible or inaccessible as a regular user. Instead, it will simply report that it is capturing "Private Folder" or "Adobe Photoshop.app".

Also—and this is the experimental part—it will no longer collect icon or localized display information about these files. The advantage is a (marginal) capture performance improvement. But it also means that if you enable the viewing of invisible items or packages in the archive browser, you won't see pretty icons or localized names for those items.

I'm fine with the package part of this. If a package is large, like a large app or a virtual machine file, I probably already know it's large, and I probably rarely, if ever, see the contents inside the package. So if I see in the activity window that, say, "Windows 10 x64.vmwarevm" is taking a long time to back up, that matches my expectations.

I'm less sure I like that behavior with invisible folders. I may be misunderstanding your intent here, so let me ask a direct question. If I'm backing up my entire hard drive, then am I just going to see "Private Folder" for the entire time while it backs up /bin, /cores, /dev, /etc, /home, /mach, /net, /opt, /private, /sbin, /tmp, /usr, and /var? If that's correct, that would NOT match my expectations. If I saw "Private Folder" listed in the activity window while it did all that, I'd probably be wondering what was wrong.

That said, I'll admit that many users probably have no idea those invisible folders even exist. Maybe most people would be more disturbed at seeing a list of folders go by that they didn't even know they had. Also, I can't recall what the current QRecall actually does with top-level invisible folders, so I don't know if what you're proposing is actually much of a change or not.

Finally, if you like this kind of detail there are two new advanced settings: QRMonitorOpaqueFolderProgress and QRMonitorInvisibleFolderProgress. Set them to true to enable the progress reporting of those items (although this doesn't change the display metadata collection).

Sounds like the new settings will allow me to get exactly what I want (once I figure out what that is. ) I'll definitely have to play with the new settings, and I'll let you know what I think after I've tried it out.

-- Bruce

James Bucanek wrote:While I can't do anything about Apple's control design, you'll notice that the collapsed version of action (click to toggle between the two) has a new-and-improved background progress indicator of my own design, which I hope clearly distinguishes between these two states.

Oops, I didn't really look very hard at that one. The first time I saw the new Activity Window, I immediately decided I wanted more detail, so I clicked on it to bring back the "old" style window. I just now ran a backup of my home directory, hoping to see the new progress indicator in action, but it immediately jumped to about 95% complete, and then stayed there for most of the rest of the time until the backup completed, so I still didn't get a good look at the new indeterminate indicator. I'll keep watching for it.

I've looked that this code a lot and don't see why it should be doing that. Having said that, I've recently made performance improvements in the code that updates the activity monitor with the status of the running action; one side effect is, I hope, more timely updating of the actual status. See if the next (post 2.1.0b32) version improves anything.

At first I thought it might be because I'm backing up a folder that contains only a single enormous file -- the VMware virtual machine file. But that file is actually a package containing many files, only one of which is screenshot_0000.jpg. There are also a bunch of hard drive "chunks" inside the package, and I never saw any of those listed in the Activity Window, but they did get backed up. So I'm not sure what's going on there either.

-- Bruce

Well, it's two and a half years later, and now I'm testing beta 2.1.0.32, but I thought I'd note that both things I mentioned in the previous reply are still issues:

1) The indeterminate progress bar in the Activity window is still nearly impossible to tell from a 100% full progress bar. I realize that's an OS issue, not really a QRecall issue, but Apple has apparently decided this isn't enough of a problem for them to do anything about it.

2) I'm doing a new backup, to a new archive, of my VMWare Fusion virtual machine file. It's been running for about 42 minutes so far, and backed up about 130 GB, and for the entire time (except for perhaps the first few seconds), the Activity window says "Capturing screenshot_0000.png", which is only a 1.9 MB file. I'm sure it's backed up way more than that, but it just seems a little odd. Is there anything you can do in the new version of QRecall to help with that?

-- Bruce

I've been using QRecall for years, as well as being a beta tester. I've used it on various Macs, including an Xserve running Snow Leopard (still in operation), all the way up to a late-2015 iMac running El Capitan. I've done entire drive restores several times, including at least twice on the Xserve. I continue to restore individual files on the server on a regular basis (several times a month), as users accidentally delete them. I use the latest version of QRecall to back up all my own Macs, at home and at work. In all that time, I've never lost a single file due to any problem with QRecall.

(Note that the XServe is still running QRecall 1.2.3.8. I expect to replace it with a Mac Mini Server and the latest version of QRecall sometime later this year.)

-- Bruce Giles
IT Manager
Process Engineering Associates, LLC
Oak Ridge, TN

Hi James,

I'll add to the congratulations! Also, I wanted to suggest that since discussion of QRecall comes up from time to time on the Macintouch web site, and it seems to have a number of fans there, you might want to send them an announcement about v2. According to the posting page, "Press releases should be emailed to macintouchpr at gmail.com."

I ran into an interesting problem today. I don't think it's specific to the QRecall beta, but maybe there's something you can do with the beta that will help?

I was looking through my system log this afternoon, for something unrelated to QRecall. When I opened the log file in the Console app, what I discovered, alarmingly, was thousands upon thousands (by this time, it may well have been millions) of lines like this:

12/23/15 1:31:40.001 PM com.apple.xpc.launchd[1]: (com.qrecall.scheduler.agent[19322]) Could not find and/or execute program specified by service: 13: Permission denied: /Users/bgiles/Library/Application Support/QRecall/QRecallScheduler

12/23/15 1:31:40.001 PM com.apple.xpc.launchd[1]: (com.qrecall.scheduler.agent[19322]) Service setup event to handle failure and will not launch until it fires.

New lines were being added to the log file at a rate of just under 300 per second! I should note that QRecall wasn't the only program doing this. 1Password was doing exactly the same thing, at the same rate. The computer had been on for less than 4 hours when I discovered the problem, and already my /var/log folder was all the way up to system.log.786.gz.

Here's what happened, and how I fixed it.

A couple of weeks ago, I retired my 8-year-old 24" iMac, which originally had the QRecall beta on it. My home directory on the old iMac was called "bgiles". I also had a newer (only 3 years old) 27" iMac which I was repurposing to replace the old iMac. It also had a home directory called "bgiles", and it had also been running the QRecall beta. Both iMacs were running the latest El Capitan OS. I used Migration Assistant to transfer all the files from the old iMac to the new iMac. Migration Assistant will not merge two home folders, so it asked me to rename the account that I was moving from the old iMac. I chose the name "bgiles24". (James, you can probably already guess where this is going.) The transfer completed without problems, and now the newer 27" iMac had both a bgiles and a bgiles24 account, each with a separate home folder. My intent was to (eventually) clean out the "bgiles" account, transfer anything there that I wanted to keep to the "bgiles24" account, and then delete the "bgiles" account. I haven't had time to do that yet, so I had mostly been ignoring the "bgiles" account and logging in on the "bgiles24" account.

I should also note that I haven't hooked up the old external drive to the 27" iMac yet, because the external drive I had been using with the old iMac has a FireWire 400 interface, and the 27" iMac doesn't. So I hadn't actually run QRecall yet since doing the transfer from the old computer. If I had, maybe I would have noticed the problem sooner.

The problem, of course, was that QRecall was trying to talk to the scheduler, in a subfolder of "bgiles". Normally, this is no problem, but because Migration Assistant forced an account renaming, the folder that QRecall should now have been looking in would have been in the "bgiles24" path, not the "bgiles" path. QRecall had no way to know this, so it tried to look in the wrong folder, which it now had no permission to access. The result was error messages galore!

Once I realized what the problem was, I went to /Users/bgiles24/Library/Launch Agents and deleted the com.qrecall.scheduler.plist file. (It occurred to me later, that I could have just edited the file in BBEdit.) While I was there, I deleted a 1Password file in the same folder. I rebooted, and confirmed that the error messages were no longer being created, by either QRecall or 1Password. I then launched QRecall, and it created a new scheduler.plist file in the bgiles24 path that pointed to the copy of the scheduler that was also in the bgiles24 path. Problem solved. I then cleared out the hundreds of no-longer needed system.log files in /var/log.

Interestingly, 1Password hasn't recreated a file in the LaunchAgents folder, even after uninstalling it and reinstalling it, but it seems to be working fine nonetheless. As does QRecall, though I still can't do a backup until I get either an interface adapter, or a new drive.

So, everything appears to be back to normal, but is there anything you can do in QRecall to keep from generating so many error messages so quickly? Or is that the OS itself that was generating those? Had I not noticed the problem when I did, I can imagine that I would have run out of disk space fairly soon, and I hear that OS X gets really hard to work with when it runs out of disk space.

I have not yet tried the new encryption feature in QRecall. My external backup drive was already encrypted via FileVault, and I would imagine that adding QRecall's encryption on top of that would be pointless, not to mention a performance issue. But I'm curious -- in terms of performance, how does QRecall's encryption (on an otherwise unencrypted drive) compare with FileVault? What are the reasons to choose one over the other?

James Bucanek wrote:You also might consider adding either "Ignore if Application <Your VM Software> is open" or "Hold if Application <Your VM Software> is open" conditions to the scheduler of your more demanding actions.

Ah, good point! I knew QRecall could do things like that (I'm using it to back up my VMs after Fusion quits), but it hadn't occurred to me to try your particular suggestion. I'll do that.

By the way, I won't be constrained by an ancient 2007 iMac at work for much longer. I'm about to place an order for a new 27" iMac for home (with 1 TB SSD), and then the existing 27" Late 2012 model I have at home now (which is still plenty fast) will transfer to work.

James Bucanek wrote:You have the scheduler set to run in the background (the "Start and run actions while logged out" scheduler option is checked).

And there's a good reason why I have it set that way, if only I could remember what it was. Seriously, I think there's a message thread here somewhere about it, but I'm too busy/lazy to go look for it at the moment. As best I can recall, however, it had something to do with unexpected or unwanted behavior by QRecall when I booted a machine which had been turned off for a week. I think maybe it queued up all the actions that should have occurred while the computer was off, and tried to run them all at once, or something like that. The workaround was to turn on the "start and run actions while logged out" option, and not power up the external backup drive until after I had logged in. By that time, QRecall had already determined that the backup drive was offline, and the waiting actions were ignored. Or something like that. Anyway, that was prior to the QRecall 2 betas, so may not even apply any more.

Short-term solution is to uncheck the "Start and run actions when logged out" and the scheduler will get installed as a user agent, which works reliably (knock wood).

I've now done that. Also updated to b21. We'll see how things go...

One of the computers I'm testing QRecall with is an old mid-2007 iMac, running El Capitan 10.11.1. It actually works surprisingly well, although it does tend to bog down a bit when QRecall is doing things while I'm running Windows 10 in a virtual machine, since each of them only gets 2 GB of the total 4 GB installed RAM. I've replaced the spinning hard drive with an SSD, so I assume contention for drive resources probably isn't much of a factor. Because of the slowness, sometimes I put QRecall actions on hold for a while, using the QRecall system menu on the right side of the menubar.

This morning, I had just booted the computer, for the first time in several days, and it had been a week (I think) since I last did any backups. As far as I know, there were no problems with QRecall when I last used it. (This is b20, by the way.) I knew I was going to be downloading and installing a big windows update in the VM, so I went to the QRecall system menu, intending to hold all schedules for a while. I couldn't do it, because both the "Hold All Schedules" and "Resume All Schedule" items were "grayed out". Then I noticed that the bottom item in the menu, below the dividing line below Resume All Schedules, was also grayed out and said "NSMenuItem".

I figured something that should have run at boot time didn't, so I fired up QRecall to see what would happen. After launching QRecall, I checked the system menu, and it was now working correctly. Hold All Schedules was no longer grayed out, and the bottom item was the next action on the schedule. I checked the log file and noticed that QRecall couldn't contact the scheduler and then uninstalled and reinstalled some things:

Monitor 2015-11-13 10:23:23 Minutia Cannot connect to scheduler
QRecall 2015-11-13 10:59:55 Minutia Removed resource link QRecallScheduler
QRecall 2015-11-13 10:59:55 link: /Users/bgiles/Library/Application Support/QRecall/QRecallScheduler
Action 2015-11-13 10:59:55 Minutia Install
Action 2015-11-13 10:59:55 ------- Install finished (00:00)
QRecall 2015-11-13 10:59:55 Installing system components
QRecall 2015-11-13 10:59:55 Minutia Cannot connect to scheduler
QRecall 2015-11-13 10:59:55 Minutia Uninstalling daemon scheduler
QRecall 2015-11-13 10:59:55 Minutia Installed resource link to QRecallScheduler
QRecall 2015-11-13 10:59:55 resource: /Applications/QRecall.app/Contents/Resources/QRecallScheduler
QRecall 2015-11-13 10:59:55 link: /Users/bgiles/Library/Application Support/QRecall/QRecallScheduler
QRecall 2015-11-13 10:59:55 Minutia Installed com.qrecall.scheduler.plist
QRecall 2015-11-13 10:59:55 Path: /Users/bgiles/Library/LaunchAgents/com.qrecall.scheduler.plist
QRecall 2015-11-13 10:59:55 Minutia Installed scheduler
QRecall 2015-11-13 10:59:55 version: 2.0.0b20

At the moment, QRecall is doing a scheduled capture, and everything seems to be working OK. When it finishes, if I have a few moments, I'll reboot and see if the situation repeats. I'll let you know the results, and I'll also send a report.