Pierpaolo,

There's no display mode in QRecall that will give you exactly what you're asking for.

The closest I can suggest is to set the bottom shader so the last layer (7) you are interested in is visible. Then set the view options to "Show Deleted Items" and "Show Timelines." Then look at the timelines for each item. Timelines of deleted items that end in the pervious layer (6) were just deleted. Timelines that include a tick in layer 7 were recaptured in that layer.

pirem71 wrote:P.S. I was one of the guys suffering from "extended attributes issue" (I'm using Skim) and release 1.2.0(64) completely solved the problems. Great work!

That's great news.

Ralph Strauch wrote:None of the hangs I experienced yesterday show up in the log.

There's nothing in the QRecall log (because the application was well and truly stuck), but OS X conveniently recorded some .hang files in the DiagnosticReport.

The .hang files show that QRecall was stuck trying create the document window, something that normally happens almost instantaneously. In your case, however, Lion was (inappropriately) trying to create "versions storage" for the archive's document. This is part of Lion's new document versions feature, but there are still bugs in this service that have plagued Lion since it's first release. A number of these bug were fixed in 10.7.3, but clearly not all of them. QRecall archives are not version-able documents, and Lion should never try to make versions of one.

Here's a sample from one of QRecall's attempts to create an archive window in which to start the repair:

14 __-[NSDocumentController openDocumentWithContentsOfURL:display:completionHandler:]_block_invoke_3 + 252 (in AppKit) [0x7fff8b63edcf]

  14 -[NSDocumentController _openDocumentWithContentsOfURL:usingProcedure:] + 530 (in AppKit) [0x7fff8b4d1df9]

	14 __-[NSDocumentController openDocumentWithContentsOfURL:display:completionHandler:]_block_invoke_4 + 648 (in AppKit) [0x7fff8b63f066]

	  14 ??? (in QRecall) [0x100069857]

		14 -[NSDocumentController(NSDeprecated) openDocumentWithContentsOfURL:display:error:] + 810 (in AppKit) [0x7fff8b64ba7e]

		  14 -[NSDocumentController makeDocumentWithContentsOfURL:ofType:error:] + 333 (in AppKit) [0x7fff8b641be8]

			14 ??? (in QRecall) [0x100002134]

			  14 -[NSDocument initWithContentsOfURL:ofType:error:] + 257 (in AppKit) [0x7fff8b4f78f1]

				14 -[NSDocument _initWithContentsOfURL:ofType:error:] + 41 (in AppKit) [0x7fff8b4f7977]

				  14 -[NSDocument _updateRequiresTemporaryVersionStorage] + 89 (in AppKit) [0x7fff8b620af4]

					14 +[NSDocument _temporaryVersionStorageRequiredForURL:] + 33 (in AppKit) [0x7fff8b62adc4]

					  14 GSLibraryCreateForFile + 549 (in GenerationalStorage) [0x7fff8710936d]

						14 ipc_checkin_client + 231 (in GenerationalStorage) [0x7fff870fc045]

						  14 mach_msg_trap + 10 (in libsystem_kernel.dylib) [0x7fff88d6d67a]

You can see that QRecall was stuck in the function GSLibraryCreateForFile, which is Lion's low-level document version storage management.

I don't know if QRecall would have ever come back. Earlier bugs on Lion would cause regular documents to hang for 30-40 seconds when being opened while Lion tried to create version storage for it. But with a document as large as a QRecall archive, who knows how long it would have taken to come back (hours?).

Anyway, the next time you tried this Lion had obviously come to its senses and let you create the document window immediately, choose the repair options, and get on with it.

pirem71 wrote:1. The status windows would includes buttons (one for each archive) to launch an archive verification without opening QR, opening the archive and use the menu item

That's a good idea. There's some technical difficulty in implementing something like that. Someone recently suggested an option to run any action from the status menu. Both of these features run up against the same architectural obstacles, but if I can solve that problem, both of these features could be implemented.

2. (independent form point 1) It would be nice to have an option that allows QR to perform automatically the verification based on the same logic now used for the "traffic light" (i.e. if an archive was not verified since long time, instead of switch on the red light QR directly performs the verification).

That's a brilliant idea. I'm not sure exactly how I'd implement it, but that's definitely going on the to-do list.

Please note that QRecall 1.2 is feature complete, and is cruising towards a final release soon. Suggestions, such as these, will be going into the hopper for version 1.3.

Ralph Strauch wrote:It gets curiouser and curiouser!

Indeed.

Is there anything I should do about the problems reported during the repair?

No, this perfectly normal for a repair that follows an interrupted compact action. The compact action works by copying records in the archive so there's no empty space between them. Think of pushing the beads on string all to one end. An archive might start out looking like this:

-1--23-4-5--67--8-9

At the end of the compact action, it will look like this:

123456789----------

If, however, it gets interrupted in the middle of the process, it looks like this:

12345*-4-5--67--8-9

The repair scans from the beginning and finds two problems. First, it encounters a damage record, the result of the interrupted write of record 6. Then it finds a bunch of duplicate records. All of the records have a unique identifier, so it knows they're duplicates. The repair ignores the duplicates and erases them from the archive. After the repair, the archive will look like this:

12345-------67--8-9

You'll notice that all of the original data is still there.

The interesting thing there, I think, is that that Qrecall log shows no activity between the first failed attempt to open the corrupted archive and the successful backup today.

There may have been some underlying problem that was preventing QRecall from launching the QRecallHelper tool. This would explain why no actions were run and the repair "hung" as well.

None of the hangs I experienced yesterday show up in the log.

Sadly, a hang is a stuck process, and a stuck process usually isn't doing anything at all—including logging.

I'll take a look at your diagnostic reports and see if anything jumps out.

Anyway, things seem back to normal now.

That's good.

Ralph Strauch wrote:One of the things that's impressed me about Qrecall has been it's ability to repair damaged archives when that was necessary. I may have exceeded that capability though now, unfortunately. One of my backup drives became unmounted in the middle of a compaction, and that may have become damaged it beyond repair as a result.

That, alone, will not irrevocably damage your archive. The compact action moves data in an archive around in a manner that is specifically engineered to avoid any data loss should the action be abnormally terminated. While the archive will need to be repaired, the probability that any information could be lost is extremely small (ilke 1 in 10 million).

When I try to mount it or to Repair the archive, Qrecall hangs with an SPOD and won't go any further.

There's something clearly wrong here, but it's not the repair action.

Activity Monitor shows Qrecall as unresponsive and using about 0.2% CPU.

The QRecall process (aka. the QRecall application) doesn't perform the repair. In fact, it doesn't perform any archive actions. The QRecallHelper process is the faceless process that performs all changes to archives, including repairs. The application's job is to start the background QRecallHelper process, but it doesn't sound like that's happening.

Is there anything else I can try, or should I just consider that archive to be toast?

(1) Send a diagnostic report.
(2) Launch the Activity Monitor application. Enter "QRecall" into the toobar search field (it will make it easier to find QRecall related processes).
(2a) With Activity Monitor running, start the repair.
(2b) If you get a spinning Technicolor pizza of death, select the QRecall process in the Activity Monitor and click on the Sample Process button in the toolbar (Command+Option+S). Save the results to a text file and e-mail that support@qrecall.com.
(2c) See if there's now a QRecallHelper process running.

Gary,

I've investigated the problem, but I don't have a solution.

I can tell you what's not happening. When a command process starts, it announces itself by sending a message to all listening processes. This is handled by a OS X framework called the Distributed Notification Center.

In a nutshell, the command says "hey, I'm starting, if anyone wants to monitor my progress you can do so." The monitor process listens for that message, connects with the command, and displays its progress.

On your system, the message sent by the command doesn't always get to the monitor. I filtered your log records down to just those between the monitor process and the commands. Most of the time it works the way it should, like this:

2012-02-21 06:37:47.325 - Monitor process started (pid=190)

2012-02-21 06:47:03.827 Command requested monitoring (pid=403)

2012-02-21 06:47:03.829 Monitor received request for monitoring (pid=190)

2012-02-21 06:50:23.423 Command requested monitoring (pid=406)

2012-02-21 06:50:23.426 Monitor received request for monitoring (pid=190)

2012-02-21 09:21:11.037 Command requested monitoring (pid=1031)

2012-02-21 09:21:11.041 Monitor received request for monitoring (pid=190)

The command announces itself, the monitor receives that announcement within milliseconds, and starts monitoring it.

But for some reason, the announcement sometimes gets lost. This if from the same diagnostic report you sent:

2012-02-23 00:14:54.776 - Monitor process started (pid=195)

2012-02-23 00:15:23.959 Command requested monitoring (pid=235)

2012-02-23 00:16:14.360 Command requested monitoring (pid=242)

2012-02-23 00:16:33.947 Command requested monitoring (pid=235)

Here the commands announce themselves again and again, but the monitor process never receives any announcements. This agrees with the behavior you describe; you start an action, but the monitor window never appears.

I don't see anything unusual about your configuration, nor can I explain why it works sometimes and not others. The only thing I can think of would be to look in the system.log console log to see if there are any messages related to the QRecallMonitor process around the time that the notification wasn't delivered.

I'll report my findings to Apple, but I don't have a fix for this at this time.

Ralph,

You're absolutely correct. The "ignore ownership" option completely slipped my mind.

So for the record, setting the "ignore ownership and permissions" option on the volume is the third way of avoiding ownership collisions between multiple systems.

The ignore ownership option is not a property of the volume. It is actually remembered in a database that's stored in the startup OS. If you do anything that might reset that database (like a clean install of the OS), or do something to the volume so that its UUID changes (like reformatting it), then the OS forgets that it was ignoring ownership on that volume and the setting reverts to its default, which is to NOT ignore ownership and permissions.

Ralph Strauch wrote:The iMac backup is done from an account named "merna" and the MBP from and account named "ralph."

That is most likely the problem. A QRecall archive is a document, and is subject to all the regular document ownership and permission rules. That is, you typically have write access to documents you create, but not documents created by another user.

When the MacBook Pro captures, it should log into the iMac as "merna" not "ralph". That way, the archive essentially belongs to "merna" and is updated as "merna" both on the iMac and from the MacBook Pro.

I've been running this setup since I started using qrecall in 2007 or thereabouts, and the problem is a new one, so I assume the recent version does something different in the way it handles permissions.

The handling of files in the archive bundle hasn't changed significantly in years.

I suspect that either MacBook Pro is currently logged into the iMac as "ralph" when the capture starts, or the alias and/or keychain of the MacBook Pro has recently decided that the default log-in for the iMac's volume should use the "ralph" account. Either way, when the MacBook Pro writes files remotely on the iMac, it's doing so as "ralph" which changes the ownership of the archive files.

If you think the MacBook Pro is auto-logging in as "ralph" instead of "merna", you can delete the saved password for "ralph" from the keychain (in the Passwords section, find the "network password" kind of entry for the iMac that has an "afp://..." location, and just delete it). Now disconnect from the server and manually start the capture action. When QRecall tries to auto-mount the remove volume, it will either go back to using the other saved account/password, or will prompt you for a log in. Log in as "merna" and click the "save on keychain" option.

An alternative is to capture to separate archives, and then the ownership won't collide.

Gary K. Griffey wrote:I simply unchecked the "Start and Run Actions while logged out"...and this issue has, at least so far, completely disappeared.

Good, that saves me from posting that I think this might have something to do with the scheduler process running as a system daemon.

One other question...does deleting the QRecall log from the mac's Console application...then restarting the system and emptying the trash have any deleterious effect on QRecall from that point forward?

QRecall won't care in the least.

Of course, I prefer that people not throw away their log files, because there's sometimes diagnostic gold in those mountains of text. But that's just me.

Sometimes, I like to clean out all the log entries and start fresh.

I will note that QRecall has automatic log rotation built in (QRecall > Preferences... > General). You can set it pretty aggressively, up to the point of only keeping one day's worth of log records.

Gary K. Griffey wrote:Is there any update on this issue?

This, and related issues, were addressed in QRecall 1.2.0(53), QRecall 1.2.0(58), and again in QRecall 1.2.0(59).

The changes in 1.2.0(53) should have solved, or at least improved, the monitor window problem that you're describing. I can't fix the communications failures in Lion, but the helper process is now much more aggressive about trying to re-establish a connection with the monitor window. When I start an action manually, ever once in awhile, it does not immediately appear in the activity window. But the action now knows this and retries about every 15 seconds, so after 15-30 seconds it finally appears. My advice is to leave the monitor process alone for a minute or so and see if it catches on.

The status of the actions in the QRecall actions window is not as robust. Basically, if the scheduler's announcement to the application gets lost, the application simply won't update its display until the next announcement arrives. But you could quit and restart QRecall to see if that causes it to catch up.

And please send a diagnostic report. Maybe there's some clue about what's (not) happening in the log.

Adrian Chapman wrote:Do the problems you have encountered relate to using a Zevo formatted disk for archive storage and/or for capturing files on a Zevo formatted disk?

The only thing I've testing is capturing to a QRecall archive stored on a ZFS volume. I'll try capturing and restoring files on a ZFS volume when I have a chance.

jeffry luvall wrote:I don't think switching over to ZFS will be as easy as claimed.

It may not be ready for prime time just yet, but I have very high hopes for ZFS. I think it's a fantastic storage technology and I look forward to working with Ten's Complement on getting these issues ironed out.

jeffrey luvall wrote:Can Qrecall be used with ZFS Storage Technology formatted drives?

Not at this time, unfortunately.

A few customers, and myself, have tried using QRecall on a ZFS volume using the ZEVO implementation from Ten's Compliment. QRecall uses a large number of the Core File API functions in OS X, and there seems to be some idiosyncrasies with a handful of these functions when executed on a ZEVO formatted volume—enough idiosyncrasies that QRecall won't work.

I've documented the issues that I've found and I'm working with Ten's Compliment in the hopes of resolving them and getting QRecall fully ZFS compatible.

Adrian Chapman wrote:

James Bucanek wrote:Question: Is your QRecall application on a different volume than your home folder?

Yes

Good, because that's the bug I addressed.

Gary K. Griffey wrote:This file name, however, is in reality, the last file that was captured successfully...not the file that is currently being captured.

Sometimes.

Normally, the name being displayed is the file that is currently being captured or the folder that was just captured.

The display of items names and icons, however, is complicated by a number of factors. QRecall doesn't (usually) display the name of invisible items, items inside invisible folders, items that are normally inaccessible as a regular user, or items inside packages. So it is often displaying the name of the file or folder it just finished capturing, rather than the (invisible/hidden/inaccessible/package) item it's actually capturing.

Is this working as designed?

It's working the way it was written.

I've had an item on the to-do list to display something more intelligent under these circumstances for a very long time. But given the complexity of the problem, and the benefits, it has a rather low priority.

I recently addressed this issue, but the fix hasn't appeared in a beta release yet.

Question: Is your QRecall application on a different volume than your home folder?

This message appears when the activity monitor process fails to find the QRecall application bundle. The QRecall bundle contains the Sparkle upgrade framework, which the activity monitor uses to determine if a QRecall upgrade is available. If the activity monitor can't locate and load the framework, the worst thing that will happen is that an upgrade notification won't appear in the activity monitor window.