So to answer your question, in a roundabout way, some of the things you're doing (installing new hardware, repartitioning) will result in QRecall treating the new volume as a brand new volume. Other techniques, like block-level cloning, may trick QRecall into thinking the new volume is the same one it has seen before.

I'm aware of UUIDs and except the one time I reinstalled the system from scratch, it should not have changed since I used OS X's Disk Image tool to image and restore the system. No hardware change took place recently, either, certainly not after I tested the v1-to-v2-crash issue when beta10 came out a week ago. That's why it's a puzzle to me: same procedure, different results.

QRecall > Help > Guide > Advanced > Combine Items.

Doh! I read it months ago, and apparently forgot about it. Sorry and thanks!

Wait, the "combine" command is grayed out if I choose all 4 volumes. Only 3 of them--1 or 2 with the other two--can be combined. In other words, volume 1 and 2 can't be combined. This is probably due to the fact reported in my previous post that the two volumes have "parallel" layers (circled in red in the screenshot).

Just read the "Limits of combining items" in the Help, and found a note about volumes with overlapping captures can't be combined. I guess that's it. The problem is: volume 1 and 2 shouldn't have overlapped.

Looking closely, I found the "overlap" was from 7/4 to 8/1, and 8/2 might be the day I started trying out beta 10. (Not sure, but likely.) If that is true, then does it mean both b10 and b11 thought, when making their first backup, they were continuing the volume started on 7/4 (the day I reinstalled my system from scratch), and yet b11 thought, at the same time, the volume has changed from the previous day. How is that possible?

James Bucanek wrote:QRecall's scheduler is managed by the launchd service. launchd is set to run the scheduler all the time, unless the QRecallScheduler process exits with a non-zero status. launchd will also throttle, or eventually stop running, the scheduler if it repeatedly crashes or is forcibly terminated too often.

At the time of my posting, there was one "Unexpected problem; scheduler stopping immediately" error in the log, and I force quit it once (in Activity Monitor). They are hours apart. I doubt that qualifies as "too often".

Relaunching the QRecall application won't help, because it's launchd's run the process.

Is it possible to have a "Restart Scheduler" command on QR's menu, perhaps next to the "Restart Monitor" command (which, btw, is almost never used since the Monitor always restarts itself very quickly)?

Ming-Li Wang wrote:1. I used "sudo qrecall captureprefs list -r -s /" to check the QR Capture Preferences settings, and it aborted with the following error midway through the process:

* failed to read directory

This isn't surprising, because you don't have permission to read those directories. Use sudo qrecall captureprefs list -r -s / instead.

I did use "sudo", didn't I?

Hmmm. The "scheduler stopping immediately" error might be the reason your scheduler isn't running...

I doubt it. As said, the scheduler went away and never came back only after I force quit it. The error in the log happened several hours earlier without my immediate knowledge. The scheduler apparently restarted itself without issue; otherwise I would have nothing to force quit.

I have an archive for my main system partition which was started on 1/31 this year. I've since reinstalled my system from scratch once, and reverted my system back to an earlier state numerous times, though the name of the partition never changes. (The arrangement was described in greater detail in an earlier post at http://forums.qrecall.com/posts/list/522.page#2419.)

Throughout the time I've always backed up the system partition to the same archive. For unknown reasons QR sometimes considers the system to be "new" and recaptures everything. The "full" backup takes longer, but doesn't take more space than it should thanks to QR's excellent deduplication capability, so I don't really mind.

Today QR did it again after I 1) imaged my system partition by booting into the rescue partition, 2) restored one of my earlier system image with QR v.1.2.3 installed, in order to test if b11 would crash after installation (no, it didn't), 3) restored my system with the image produced in step 1, 4) installed QR v2 b11 and 5) ran the system capture action.

As a result, I have 4 sets of captures for the same volume in the archive (see attached screenshot below). Look closer and you'll see the newest set (set no.1) has another "incarnation" before 8/1. Odd, isn't it? Furthermore, set 1 and 2 have the same layers from the start to 8/1 (in the red circle). How is that possible?

Another oddity: I went through the same steps described above to test b10 about a week ago, and yet b10 didn't consider the system "new" and continued to backup to the old set (set 2). How come b11 did it differently?

While I don't particularly mind QR starting a new set from time to time, this practice does have its down side: when digging through the archive for some earlier versions of a file, I can't reach all versions of the same file easily.

Hence, a question: is there a way to "merge" the sets? If not, please consider it a feature request. Thanks!

Most of the issues I reported earlier have been fixed as stated in the release notes. Thank you!

One problem remains: I still can't restart QR scheduler after forcing it to quit in the Activity Monitor. Quit and restarted QR several times, but the scheduler never came up. A diagnostic report has been sent.

The issue isn't mentioned in the release notes, I know. Just thought it might have been solved with all the changes made to the scheduler.

Other issues:

1. I used "sudo qrecall captureprefs list -r -s /" to check the QR Capture Preferences settings, and it aborted with the following error midway through the process:

* failed to read directory

This, in fact, isn't new. I ran into this as well with b10, but forgot to report it. Sorry.

2. There are a few dozen of "Unable to communicate with helper" warnings and one "Unexpected problem; scheduler stopping immediately" error in the log. The error happened more than 6 hours ago, so no report was sent. I'll send a report should I spot it sooner next time.

James Bucanek wrote:

Btw, I couldn't get QR scheduler to restart after having it killed in Activity Monitor (relaunching QR didn't restart the scheduler).

Please send a diagnostic report. There might be a clue as to why the scheduler didn't restart.

Done. I force quit both the scheduler and the monitor in the Activity Monitor right after a reboot. The QR monitor recovered in less than a sec., but the scheduler didn't. Opening QR didn't help, had to log out and log in again to get the scheduler back.

A small issue out of this test: I removed the test archive after finishing the test, but forgot to remove the scheduled capture actions associated with it first. QR failed to execute the capture action as a result. I've since removed the action but now I get two ghost items in the QR pop-up menu that refuse to go away. (screenshot attached below)

Note in the screenshot that the two ghost items were scheduled for 0:00, but the system time was at 0:28 already. Clicking any of the item on the second pop-up (Stop, Stop and Reschedule, etc.) does nothing. Guess I'll have to reboot to get rid of them.

If one make a change to the number for the "Keep deleted items for at least ___ [unit]" setting in the archive settings window, then click the "Cancel" button to dismiss the window, the old value would not be retained as expected. Instead, the new value would be kept.

When the Archive Settings window is opened, in addition, the focus (cursor) rests in the number box with the whole number highlighted. If one hits the ESC key at this time (in an attempt to cancel the dialog), the number would be replaced with the letter "I". Hitting ESC again would change it back to the old value, but clicking outside the box would make the "I" stick and get an error message about invalid number.

I've since managed to reproduce the problem with different archives and on different disks/partitions. I'll email you a test archive that crashes QR when compacted (at least on my system, every time) in a minute.

Ming-Li Wang wrote:6. I can't disable the "Cache Files" standard exclusion for one of my archives. I've unchecked the item and saved, many times over, but it always remains checked the next time I open the archive settings window. Very weird, for there is no such problem for other archives.

I spoke too soon on this one. It seems the choices made about standard exclusion options on the "Archive" - "Settings" page don't always stick.

For example, if an archive has all but "Cache Files" checked, then you uncheck the "Items Excluded by Time Machine", the choice would be undone once you close the archive.

I made a 30-sec. screencast (.mov file) to show you exactly this. In the video, I

1. opened an archive named docu, opened the Archive Settings window (with the keyboard shortcut), and unchecked "Items Excluded by Time Machine",

2. opened the Settings window right away, and as you can see, the "Items Excluded by Time Machine" is disabled,

3. close the archive,

4. open the archive again, and

5. open the Settings window again.

Voila, now you can see for yourself that the "Items Excluded by Time Machine" option has been reverted back to enabled state.

I can't find any pattern yet except:

1. Clearing all options (disabling all) always sticks.

2. Enabling any combination of options (that I've tried) when none was checked (i.e., after taking the previous step) always sticks, too.

Perhaps I should add: the archive was created by v1 a few months back, and is updated often. Since I switched to v2 yesterday, 9 layers have been added. Redundancy has been enabled, too.

I made a backup copy of the archive before going v2, which can be compacted without issue.

Got "internal program error" midway through a compact action. The Log shows "unexpected buffer contents" messages. Report filed.

Couldn't open the archive afterward; had to repair it first. Verified the volume first and ran into incorrect file hard links numbers (or something like that) error. The disk was repaired successfully and then so was the archive.

Tried to compact again, and ran into the same error again, only this time disk verification turned up no error and the archive was successfully repaired.

Tried to compact other archive and haven't run into similar trouble so far. Could be something wrong with the disk, or the archive.

James Bucanek wrote:That's what I see too. But the OS X crash logs are remarkably uninformative in this case, so I can't tell exactly where/why it's crashing. If you have copies of those old (pre-upgrade) action documents, could you please email them to support@qrecall.com?

Done.

Actually it's modifying a whole bunch of things. And I don't put everything in the release notes; they're long enough as it is.

Understood. Your release notes are indeed very informative. Thank you.

One of the things that gets updated for every action are the alias records.

I didn't know anything about "alias" in QR, and therefore have never used it intentionally. Is it the same thing as Finder alias?

Some more observations and thoughts:

1. The "+" and "-" buttons for customized exclusions are indeed working now, thanks!

2. The "qrecall" command line is very useful. I can now set up all my permanent exclusions in a bash script. Wonderful.

3. I turned on redundancy for a few archives. The warning (that this is a time consuming task) looked scary, but it turned out to be much faster than expected. It took merely 19 sec. to add redundancy to smallest archive (6+ GB, with many small files) on an SSD. A much larger archive (60+GB) on an slow WD Green HD took 18 min., and my largest archive (350+GB, on the same HD) took barely over 1 & a half hours (all using the 1:8 ratio). Impressive.

4. b10 seems pretty stable, but this morning (about 9am, Taiwan time) I found the Monitor window on the desktop, listing the verification for an archive as the next action, but the scheduled time for that action was 4:40 am. QR was very sluggish. The rainbow wheel spinned for a few sec. with each move (clicking on the menu icon, clicking a menu item, etc.). Restarting the Monitor didn't help. I had to kill all QR processes in the Activity Monitor. The log shows a lot of "Unable to communicate with helper" warning since 4:10 this morning.

Btw, I couldn't get QR scheduler to restart after having it killed in Activity Monitor (relaunching QR didn't restart the scheduler).

5. Starting an action manually in the "Actions" window doesn't start it immediately. See the attached screenshot--next action at 11:39, system time at 11:43--for an example. The action was finally executed at 11:45, which was the nearest time for a scheduled action (for another archive). It's the same when I tried to execute another action (a Verify action) manually.

6. I can't disable the "Cache Files" standard exclusion for one of my archives. I've unchecked the item and saved, many times over, but it always remains checked the next time I open the archive settings window. Very weird, for there is no such problem for other archives.

7. Speaking of cache files (and items excluded by Time Machine), v2 has retained the same behavior from v1 of excluding the whole "Caches" folder (not just its contents). Would you consider changing it to something equivalent to the new "contents exclusion" mode? Time Machine's standard exclusion plist specifically says some folders should be retained (while their contents are discarded), and my experience was the system would behave abnormally if some of the folders were not there after restoring from a backup.

8. When trying to check my QRecall Capture Preferences settings with the "qrecall captureprefs list -r -s ~/Documents/" command, I got the following error message:

2015-08-02 12:07:28.975 qrecall[8920:60370] *** Assertion failure in char *DDNewCPath(const char *, const char *)(), /Users/james/Development/Projects/Quantum Recall/Common/Source/Filesystem/DDPath.m:696
* parent path (/Users/wang/Documents/) ends in '/'

Removing the trailing slash solved the problem, but you might want to add some fault tolerance codes or tweak the error message a little to make it less scary.

That's all for now. Thanks for listening.

OK, more interesting findings.

1. Before I rebooted (for the first time after b10 installation), I moved all the actions back into the "Actions" folder, and tried to open QR again. Voila! It opened without problem. Open the Actions window and I could see all the actions. No conversion was done.

I rebooted and reinstalled b10 again, with my old Actions folder in place, and QR launched without problem after installation, but again no conversion was performed. I thinks it's because the 2nd installation checked and realized v2 has been installed before, and skipped the conversion step.

I've since done more tests, and can confirms that v2 would crash on my system after installation when upgrading from v1 to v2, unless I empty the "Actions" folder beforehand.

What I don't get is: v2, on first run, should only try to convert customized exclusions into QRecall Capture Preferences, according to the release notes. If that's the case, only capture actions should matter. And yet, as stated in my previous report (see step 2 in the previous post), removing all "capture" actions from the Actions folder didn't prevent QR from crashing after installation. Only removing all actions worked.

2. On the missing QRecall Capture Preferences service, I've found that it (along with 2 other services for "Files and Folders") would be installed only when v2 is installed after every files and folders related to QRecall has been removed before installation. Before doing that, I tried removing all the plist files inside ~/Library/Preferences, the QRecall folder in the same folder, and all plist files inside the various LaunchDaemon/LaunchAgent folders on the system, and still couldn't get those services installed.

That's all for now. Off to play with the new beta (thank you for the speedy release).

Installed beta 10 just now and it crashed right after upgrade, same as my experience with b9.

Since I knew it had something to do with defined actions, I tried the following in order:

1. removing 2 of the capture actions I considered to be most likely offenders since they have many exclusions defined, some of which don't exist on the system at the moment.

2. removing all capture actions.

3. removing all actions from the "Actions" folder.

QR finally launched without issue after the 3rd step. A diagnostic repot was then sent immediately.

Oh, I also checked and could not find the QRecall Capture Preferences service.

I'm going to reboot my system and see if that makes any difference.

It would appear that your initial install of QRecall 2.0 failed spectacularly. If you still have the QRecall.log for that time period, please forward it to support@qrecall.com. (You'll want to compress it first, as it can often be rather large.)

That's my guess as well, but I reinstalled several times, and tried my best to do it as cleanly as possible. A little more detail:

I've kept the habit from my Windows days of keeping a detailed system setup diary when setting up a new system--a plaintext file recording all the tweaks and software installations in the order they take place, along with software license codes and other notes. The installation process is organized in stages, with a fresh new system with nothing but the most basic, essential tweaks made and no software installed as stage A0, and it moves from there to A1, A2, A3 ... and so on. At the end of each stage, a system image (.dmg file) is made (by booting into the rescue partition or a live USB drive I keep for this purpose).

This way, I could revert the system back to an earlier stage if something goes wrong in a later stage. Some of those .dmg files are eventually discarded to save space when the system seems to be running without issue, but some are kept so that I can try new software at will.

When I first installed QR 2.0 beta 9, I did it on my then current, running system. When QR crashed after launch, I suspected failed installation because QR 1 was running at the time. I then tried to quit all QR processes in Activity Monitor, but one process was very persistent. I then booted into my live USB drive, removed all QR launchers in the various LaunchDaemon/LaunchAgents folders, and booted back into the system without QR running. Reinstalled beta 9 and it still crashed.

I then reverted the system back to A3, A2 and finally A0 (disk image for A1 has been discarded). Because QR v1 was installed in stage A1, A3 & A2 got the same treatment after restoration so they booted without QR running. And yet beta 9 crashed right after installation as well. And the QRecall Capture Preferences service was never installed. The only time I could install b9 without issue was when the system was reverted back to the bare minimum A0.

Sorry, no logs were kept, but I'm pretty sure the problem is reproducible on my system. If it happens again with b10 (if I have time to try it), I'll keep the log.

To set the capture preferences for an item you don't own, use the sudo and qrecall command line tools:

sudo qrecall captureprefs exclude /Path/to/excluded/item

Great. Thanks.

If it was easy, I'd do it myself.

OK. I'll see if I can run my saved systems in a VM (Virtualbox).