Sorry to hear you're having so many problems, but I suspect the manual un-install fixed whatever problem was preventing your privileged helper service from installing.

Mats Jacobson wrote:I attempted a detailed uninstallation followed by a new fresh installation after a restart and while I now get QRecall 2.1.8 properly opened I lost my schedule Actions for the main archive. Are these possible to recreate? I still have most files I think.

Each QRecall action is stored as an action file (.qraction) in the ~/Library/Preferences/QRecall/Actions folder inside your home folder. Each file is automatically named (e.g. important stuff-compact_0.qraction) in a way that should make it easy to identify what that action is.

If you have copies of your action files (say, in the archive that captures your home folder or still in your Trash), simply move/copy/restore them back to that folder and restart your system.

If you're a little bit geeky, you don't have to restart your whole system. The QRecall app and the background scheduler look at the Actions folder when they start up and after you make any changes to your actions. But if you manually alter the files in the Actions folder, you'll need to nudge QRecall into loading the Actions folder again. The quickest way to do that is to re-launch the QRecall app and the QRScheduler process. The later can be restarted using the Activity Monitor app; use the search field to find the QRScheduler process and tell it a "Quit" (a TERM signal, if you want to do this from the command line). It will immediately restart and reload the actions in the Actions folder.

Mats,

It seems you have a weird, partial, installation. You definitely have a scheduler installed and running, but it's the 2.1.4 version (which is why QRecall refuses to recognize it).

But the real problem is that you have privileged services installed, but your system isn't preauthorized for administrative privileges.

My guess is that at some point QRecall prompted to upgrade its privileged helper service and something went wrong, so the pre-authorization went away leaving some services installed/updated and others not.

I suspect you can solve this simply by launching QRecall and going to QRecall > Preferences. Choose the Authorization tab and check the "Capture and recall items with administrative privileges" option.

QRecall should then prompt for your administrator's account and password. If successful, it should install the privileged helper service.

Count to three, quit QRecall, and launch it again. If I'm right, everything should get reinstalled and start working again.

If I'm wrong, you know how to send another diagnostic report...

Nico,

QRecall logs almost every reason it does or does not do something, but sometimes it's considered "minutia" so you might need to slide the Details level in the log window further to the right.

If you don't thing somethings working, start by sending a diagnostic report. Open QRecall and choose Help > Send Report from the menu.

That will collect your recent log activity (a lot of which you don't normally see), crash reports, and configuration information that will hep us figure out what's going on.

Mats,

Let's start by getting a diagnostic report. Open QRecall and choose Help > Send Report from the menu.

That will collect your recent log activity (a lot of which you don't normally see), crash reports, and configuration information that will hep us figure out what's going on.

Nico,

QRecall can certainly help you. Although, if I'm being completely honest, you don't need QRecall to accomplish this.

Here's how I'd do it using QRecall:

(1) Format and install Sierra on an external drive.
(2) Download and install QRecall
(3) Using the capture assistant to create an archive on the external drive and capture your entire startup volume. (Have a snack while QRecall does its thing.)
(4) Boot from the external drive
(5) Use Disk Utility or whatever you need to replace/erase/reformat your internal drive.
(6) Download and use the High Sierra installer to install a fresh OS your empty internal drive.
(7) Download and install QRecall again (or just drag the copy of QRecall from the external drive on to the internal one).
(8) Open QRecall and open the archive on your external drive.
(9) Selectively recall the items you want. You can do this simply by dragging an item from the archive into Finder to re-install or replace the items on your new system, exactly as you would using the Finder.
(10) Enjoy

Notes:

- In step 9, QRecall's "Restore" command won't work because your internal volume is brand new and is no longer the source of the items in the archive.

- Make sure that the "short" (UNIX) name of your new account(s) matches your current account(s). If you do that, Qrecall will automatically fix up any ownership issues when you recall the items so they continue to belong to the same users (something a clone or straight copy utility won't do).

If you want to continue using QRecall, you're already done 90% of the work of setting up an bootable external volume for performing catastrophic recoveries. All you'd need to do now is use the capture assistant to set up regular capture strategy of your new startup volume to the existing archive on the external drive (and with QRecall's data de-duplicating, it won't even get any bigger). And then you'll probably want to upgrade the external drive so it's running High Sierra too.

Let us know if you have additional questions or run in to any problems.

Here?s one solution, that assumes you?re using Apple Mail.

First, update to QRecall 2.1.7. There?s a bug in 2.1.6 that doesn?t reliably run the epilog script after an error.

Launch the Script Editor (built-in app the comes with macOS).

Create a new document and paste in this code:

set commandSuccess to (system attribute "QR_COMMAND_SUCCESS")

if commandSuccess is "0" then

	-- the command's success is "0" (in other words, the action was NOT successful)

	-- note: if the action was canceled, QR_COMMAND_SUCCESS will be "1" and QR_COMMAND_CANCELED will be <who>,

	--	so if you want to be notified of canceled actions you'll need to add a check for that.

	-- note: that if run as a prolog, there is no QR_COMMAND_SUCCESS value, so this will never execute

	

	-- compose a message with a summary of the problem

	set recipientName to "QRecall Administrator"

	set recipientAddress to "your@email.com"

	set theSubject to "QRecall action failed"

	set theContent to "The QRecall action '" & (system attribute "QR_COMMAND_TITLE") & "' failed." & return & return ¬

		& "Reason: " & (system attribute "QR_COMMAND_EXCEPTION") & return & return ¬

		& "Archive: " & (system attribute "QR_ARCHIVE_NAME_DISPLAY")

	

	tell application "Mail"

		-- create the message

		set theMessage to make new outgoing message with properties {subject:theSubject, content:theContent, visible:true}

		-- set a recipient

		tell theMessage

			make new to recipient with properties {name:recipientName, address:recipientAddress}

			-- and send the message

			send

		end tell

	end tell

end if

Edit the value of the recipientAddress at a minimum, but feel free to customize the name, subject, and content values to suit your needs.

Save the file as a compiled AppleScript (.scpt) file.

In QRecall, open the actions you want to be notified, by email, when they fail. Click the ?Select? button in the Epilog script and select the AppleScript file you just saved. The ?Needs UI? option should automatically get checked, but just make sure it is. Save the action.

That?s it.

After each edited action runs, it will execute this AppleScript which will see if the action finished successfully, or at least non-fatally. If it was successful, it does nothing. If there was a failure, it opens the Mail app, composes, and sends a message to the address in recipientAddress.

Excellent question.

And the answer is a, resounding, "absolutely!"

In fact, you can use almost any mix of filesystems.

Unlike backup solutions that simply copy files, QRecall stores the information about your captured files in a database. So it doesn't matter what the capabilities of the destination filesystem are, as long as it can hold an archive.

This makes QRecall unique in its ability to capture items from a filesystem that supports user permissions, hard links, extended attributes, access control lists, and compression, and faithfully preserve them in an archive on a volume that doesn't support any of those features.

David Ramsey wrote:I did try closing the "QRecall Monitor" application to prevent this from happening; however, closing it is impossible: that is, you can close it, but it just pops right back up.

In 2.1 the monitor app now appears in the dock and, more or less, acts like a regular app.

If you don't want it in the dock you can hide it (See QRecall > Preferences > Monitor, "Show in dock" setting).

However, if you hide the monitor in the dock you'll also disable the feature where QRecall will prevent your system from restarting or shutting down until all of your running QRecall actions have finished.

If you want to restore the pre-2.1 behavior, go to QRecall > Preferences > Monitor, turn off "Show in dock" and turn "Show in menu bar" back on.

It's possible the Monitor apps is just a monitor.

It is.

However, if it's involved in actually starting scheduled actions, it sure would be nice to close it and have it stay closed.

The QRecallScheduler process starts scheduled actions.

David Ramsey wrote:It seems to me that scheduled tasks should never interrupt a currently running task.

It won't. QRecall will only allow a single action to modify an archive at a time. And it doesn't matter how the action was started (manually, scheduled, from the command line, etc.)

A capture action will wait until all processes that are currently using the archive have closed it before it begins. Having the archive open in a browser window counts as one process (and is why you got that dialog). The second recall process counts as a second, and the capture action would have waited until both had finished before starting.

I can't think of anything for you to test, but I do have a question. Are you capturing the entire volume/folder, or specific items inside of the "Sound room" folder?

Johannes,

It looks like you're doing everything right, so I'm not sure why it's not working for you.

I just set up a similar test, capturing a folder on an external volume, excluding all of the *.dmg files, and it worked as advertised.

So here are my suggestions:

Make sure the path is correct

If you're typing in the path, it's easy to mess it up. Instead, select the path portion of the pattern, clear its contents, and then locate the folder in the Finder and drag that into the path field (the insertion cursor must be active for this to work). That will alway paste in the exact path to the folder.

Make sure the extension is correct

Select one of your .mp3 files, choose Get Info in the Finder, then look at the Name & Extension section. Make sure the extension is correct and it's really the extension of the file. (Some files have what look like extensions, but their extension is hidden in the Finder and the "extension" you see is actually part of its name.)

Verify where you are creating the pattern

You didn't mention whether you were setting this pattern in the Archive's settings or in the Capture Preferences of an item. When using the capture preferences, absolute paths are not allowed and if you create a pattern with an absolute path, it will be ignored. So you have the choice of adding the pattern to the archive's settings or using a relative path in a pattern attached to a folder:

Archive Settings: path=/Volumes/WorkingMedia/Sound room
Capture prefs for /Volumes/WorkingMedia/Sound room: path=(empty)
Capture prefs for /Volumes/WorkingMedia: path=Sound room

Finally, you also have the option of creating a .qrecall_exclude_patterns.txt file in the "Sound room" folder. It would consist of a single, simple, repeating, glob pattern:

//*.mp3/+

Let us know if any of that helps.

Paul,

The problem is Mojave. Even when you grant the helper app Full Disk Access, sometimes the operating system still denies access to protected items.

Note: first make sure you've granted the QRecallHelper app Full Disk Access. Between QRecall versions 2.1.4 and 2.1.5 the helper has changed from an executable file to an executable bundle. If you've granted the file Full Disk Access, remove it and replace it with the app. The Capture Assistant will walk you through the steps.

It seems to be a timing issue. If the helper immediately attempts to access a protected item, it will typically get blocked. But if it first reads a bunch of other files and then tries to access the same item, the request is granted.

Consequently, initial and infrequent captures tend to work just fine, while frequent ones will often fail. The reason is that when only a few items have changed since the last capture, QRecall jumps right to those items. And if one of them is protected, it access is denied.

As a workaround, I've found that setting the "Deep Scan" option of the capture action helps. It significantly increases the capture time (because QRecall checks every directory), but it's enough busy work to skirt around the bug and capture all of the files.

Naturally, I've brought this to Apple's attention and have had some short conversations about it. But at the moment, I'm waiting on Apple for a solution.

A 1.6TB archive is pretty big for being hosted on a NAS. The problem isn't so much the size of the archive, as much the number of records you've accumulated and the nature of networked volume transactions. It's also not a memory issue.

The early part of the repair (checking archive data) is highly optimized to read large chunks of the archive, sequentially. This can be very efficient even on a networked volume, and is usually constrained only by the network's speed.

Some of the later phases, like reindexing the layers, is highly transactional. QRecall is going back to each file and directory record and reconstructing the tree of items captured in each layer. It reads these (small) records one at a time.

Most filesystems are pretty smart about this, and the operating system will both cache data (so that reads of multiple, small, records come from RAM) and read ahead (so that it's likely the next read request can be satisfied from RAM). Networked filesystems tend to be transactional, which means that every tiny read request gets packaged up as a request, sent to the server, the server performs that request, and returns the results. Wash, rinse, repeat.

This means that the latency of requests for networked storage is much higher than for local storage. You can probably see this in the Activity Monitor. The total network throughput will be rather modest, but you'll see that the number of network requests per second is in the thousands.

You have a some 26,000,000 file and directory records to read and process. If you're processing only 1,000 records per second, that's at least 6 hours just to read them all. If your latency is higher, you might only be processing a few hundred a second, which could mean 18 or 36 hours. In contrast, a local drive would typically be able to read 2,000 to 3,000 records per second.

I'm working on future optimizations so that QRecall can group multiple record reads into a single request, which I hope will speed up networked volumes. But for now, it relies on the native filesystem drives for whatever optimization it can get.

In conclusion, if the repair is once-in-a-blue-moon occurrence, you might just live with the performance for now. Another option might be to split up the archive. I have a Mac mini that I capture the media (iTunes) folder to one archive and a second archive captures everything else (by capturing the startup volume but excluding the iTunes folder). These archives are stored on an Airport time capsule, and it keeps the size of each archive down to a manageable size.

Paul,

QRecall is, for the most part, compatible with Mojave. There are still issues with Full Disk Access, which we're currently working with Apple to resolve, but you've taken the correct step of adding the ~/Library/Application Support/QRecall/QRecallHelper program to the Full Disk Access group.

There are some minor bugs you might encounter. Mojave makes some subtle changes in how timers work and you might experience issues like the QRecall scheduler sometimes won't actually schedule anything for about 15 minutes after you first boot the system?oddities like that.

Mojave doesn't really change how the archives are accessed, so if you're getting "waiting for archive" status messages then maybe something else is going on. You might start by sending us a diagnostic report (Help > Send Report) and take a look at the troubleshooting topic in Help > QRecall Help > Guide > Troubleshooting > Problems > Can't Open Archive. (Note that opening an archive to modify it and opening an archive to read/browse it are different modes, and one can still work while the other is stuck.) Often just waiting 10 or 15 minutes, or a restart, will break these kinds of deadlocks.

The big outstanding issue is that Mojave's Full Disk Access doesn't act consistently. Even if you grant the QRecallHelper program Full Disk Access, some future captures might still be unable to read restricted items. Keep an eye out for an update that addresses this.

Jeff,

I agree that the assistant isn't that flexible, and is really only meant to cover very generalized solutions.

But you're on the right track: use the assistant the get a feel for what actions you need and how they are scheduled. After that, edit the daylights out of them.

The maintenance actions (merge, verify, ...) apply to everything that's in the archive, so you only need one of those. You'll probably want to fiddle with their settings, schedule, and conditions.

It's not uncommon to have multiple capture actions; either actions that capture different things or in response to different events. For example, I have three capture actions on a main development system: capture my home folder every 2 hours (ignoring changes to mail and iTunes), capture the entire volume once a day, capture important development directories 1 minute after they change.

Also note that you can attach a executable script to start or finish of any action. So if you need a special script to mount a volume (assuming it's not auto-mountable) you can write a script that will do that before the action runs. You can also have the action wait until a volume is mounted. There are lots of possibilities.