I know from experience that system configuration problems can become a Gordian knot, from which a clean break is often the only expedient solution. I'm glad to hear that you've disentangled yourself from the problem.

Steven M. Alper wrote:However, at that point all the complaints that used to be about 501 were now occuring about 503.

Did you uninstall QRecall as user 503 before renumbering the account? If the kicker was installed for users 501 and 503, changing 503 to 501 would only replace one non-existent user with another. You can still follow the instructions I gave at the beginning of this thread.

I'm not sure my problems are solved, though, because QRecall is now complaining about not being able to create files:

3/17/10 1:13:19 PM QRecallScheduler[100] 2010-03-17 13:13:19.404 -0400 Failure could not open standard log; logging to stderr

3/17/10 1:13:19 PM QRecallScheduler[100] 2010-03-17 13:13:19.413 -0400 Failure Cannot create support folder

3/17/10 1:13:19 PM QRecallScheduler[100] 2010-03-17 13:13:19.415 -0400 Details Folder: /Users/trashed/Library/Logs

3/17/10 1:13:19 PM QRecallScheduler[100] 2010-03-17 13:13:19.491 -0400 Details NSFileOwnerAccountID: 507

3/17/10 1:13:19 PM QRecallScheduler[100] 2010-03-17 13:13:19.524 -0400 Details NSFilePosixPermissions: 448

I think you've got other problems.

I don't know what user this is being reported from (I'm assuming 501), but the log is telling you that QRecall is trying to access its support folders (Logs, Preferences, Actions, ...) in /Users/trashed/Library and it can't because the folder is owned by user 507. That's neither 503 nor 501. This is really odd, because if you're logged in as user 501 then it should be using user 501's home directory. If its home folder really is /Users/trashed, then it contains items owned by user 507 ... whoever that is. Are you sure your old account's UID was 503 and not 507? If user 501 has a different account name, then the account's home path is incorrect.

In general, an account should have a home path (i.e. /Users/james) and an UID (501) and all of the files within its home folder should belong to it. So something is still out of place. You might need to reassign 507 files to 501, or fix a home path, or something.

Steven M. Alper wrote:That may solve the QRecall problem, which I assume is only a symptom of the much larger problem I discovered shortly after I wrote you: because of the client uid validation problem, the existing users don't have proper permissions.

Yes, this would only solve the problem for QRecall. Any other software that's specifically installed for user 501 will still bump into the same issue.

Meaning no write permissions, despite appearing as if they do.

The write permissions of an account/user are not a property of that account. Every file and folder is owned by a user and a group (ignoring ACLs for the moment), which it uses to grant read, write, and execute access to that user, group, or everyone in general.

The problem is likely that you have files and folders which belong to the original user (501) that do not grant your current users access.

The most straight forward solution would probably be to change the UID of your principle, administrative, user account to 501, then migrating any files owned by the current UID to 501. The "new" account will now have a UID of 501 and own all of the existing 501 files along with any that it previously owned. The process for doing this should be pretty straight forward:

1 Identify the account you want to turn into user 501. Have a second, administrative, account ready. If you don't have a second administrative account, create a temporary one for the purposes of this migration.

2 Log into the account you want to change and uninstall QRecall (Shift+Option+QRecall > Quit and Uninstall), and any other software that might be a problem. You don't want a repeat of the problem you now have with 501.

3 Log out of all accounts. Log into the second administrative account.

4 In the Accounts pane of System Preferences, authenticate and right/control+click on the account that will become user 501. Choose Advanced Options.

5 In the dialog, write down the user's current UID.

6 Change the UID to 501. Don't change anything else.

7 Close System Preferences and open a Terminal window.

8 Enter the command: sudo -s

9 Type your second account's password

10 Enter the command: find / -user <old_UID_from_step_5> -print0 | xargs -0 chown 501

11 Wait for the command to finish; it will take awhile.

12 Restart

Roughly, these steps change the user ID of an existing account to 501. The chown command is then used to change the ownership of any file or folder that currently belongs the existing account from what it was to 501. Now that account, all of its files, and any old files that already belonged to owner 501, now all belong to the one account.

Note: Normally I wouldn't use this forum for general troubleshooting, but since Dan Frakes and I are the author of the free ChangeShortName utility, I have some experience migrating, renaming, and renumbering user accounts.

Steven,

The reinstall may be the source of the problem, and some of the similar messages you've gotten from other software. My first question is "do you have a user account with a UID of 501?"

The message would appear to indicate that you do not. Since UID 501 is assigned to the first account create by the system, this might happen if you installed a new OS, create a new account, then delete the original one created by the system.

If you have no user 501, the solution might be simple. You'll need to edit the file /Library/LaunchDaemons/com.qrecall.scheduler.kickstart.plist. This file is owned by root, so you'll need an editor that can handle it, like BBEdit, or run one of the command-line editors as root (i.e. 'sudo pico /Library/LaunchDaemons/com.qrecall.scheduler.kickstart.plist '). In this file, you'll find a list of the UIDs that need a QRecall scheduler daemon:

<key>ProgramArguments</key>

<array>

        <string>501</string>

        <string>502</string>

</array>

Delete the element for 501 (and any other account that might not exist).

Background:
The QRecallKickStart daemon is a system-level daemon that spawns user-level daemons for each QRecall user that has selected the option "Start and run actions while logged out." This was done because, while OS X provides for system-level daemons, system-level agents, and user-level agents, it doesn't provide the concept of a "user-level daemon," which is what QRecall needs. The parameters in com.qrecall.scheduler.kickstart.plist tell QRecallKickStart which users have requested a scheduler daemon.

Prion wrote:Out of curiosity, if QRecall gets interrupted in the midst of dealing with one particular file it will resume its activity by starting over again at this particular file, right? It will not somehow miss the bits it has started to do but forgotten because it was interrupted?

Oops, almost forgot to answer that bit.

Every time QRecall begins a capture, it simply compares what's on the drive to what's in the archive. If it has completely captured an item and that item hasn't changed, then it skips it and moves on. Otherwise, it captures the item again.

It's simplistic logic, but it makes it almost impossible to "fool" QRecall into overlooking an item.

Prion wrote:what I meant was why enabling TimeMachine may have conflicted with QRecall, possibly the fact that both running simultaneously and/or that the USB drive is connected to the TimeCapsule may play a role here though I have not idea *how* exactly this may have been a problem.

To the best of my knowledge, Time Machine and QRecall should coexist peacefully, with some minor caveats. I have a number of customers who run both QRecall and Time Machine and haven't reported problems.

Having QRecall and Time Machine both backing up to the same drive simultaneously can result in poor performance—for both. QRecall and Time Machine both use a fair amount of memory and are very I/O intensive. The result is more competition than cooperation, resulting in wasted effort and poor performance.

A small number of customers have reported that having multiple processes hammering on the same USB drive can sometime "overwhelm" the drive or the USB interface, resulting in I/O errors. These seem to be transient, but that would definitely cause QRecall to halt in its tracks.

My suggestion is to make a note of when Time Machine normally does its thing and schedule QRecall to do its heavy lifting (like daily capture and merge actions) at some other time. You might want to suspend QRecall's scheduled actions until Time Machine is finished with its initial backup.

I definitely did not run out of space, on both hard drives there is more than 200 GB of free space available.

Then it sounds like an I/O or some other event caused QRecall to stop. The log should say. If you send a diagnostic report (Help > Send Report), I'll take a look at it.

The QR Archive on the USB drive could be repaired using the auto-repair

Auto-repair occurs when something disastrous happens (crash, I/O error) to the capture process. If the archive was auto-repaired, then I suspect an I/O error.

Whatever detail it was, everything is working again now. For now, the USB drive is connected to the MBP directly and TimeMachine is disabled. I'll await your comments on how to proceed regarding
1) coexistence of TimeMachine and QRecall

Should be OK. It might be too much for your USB drive, but it shouldn't be. Also, that could change if you move the drive to your Time Capsule.

2) connecting the USB drive to the TimeCapsule.

Should also be fine. After you move your drive to the Time Capsule, you'll need to open all of your QRecall actions and change the archive in each to refer to the new archive, which will now appear on a networked volume. You will also need to be logged in for your QRecall actions to run. Mac OS X requires an account and password to mount a networked volume, and that requires you to be logged in.

Thanks for your support! The longer I work with QRecall the more I realize just how much consideration and attention to detail went into its creation.

Prion wrote:there are several incomplete layers of capture after a certain date.

An incomplete layer is just that: QRecall stopped capturing to the archive before it was finished. It just means that one or more items modified before the date of the layer were not captured. The missing items will be captured by the next (complete) capture.

1) Any ideas what may have caused the problem technically and how to avoid this in the future?

An incomplete layer is created when the QRecall capture is stopped before it can finish. There are a variety of events that can cause this to happen:
o You stopped the capture action directly (clicking on the stop button or sending the process a TERM signal) or indirectly (log out, shutdown, restart).
o The capture ran into a problem that prevented it from finishing, like running out of disk space.
o The capture's action has a condition that automatically stops the action at some time.

If this started to occur after you enabled Time Machine backups to the same device, you might have run out of disk space. This will prevent both QRecall and Time Machine from completing their backups. Of course, QRecall tells you about this, while Time Machine just deletes data or stops.

2) QRecall can backup over WiFi, can it not?

Yes.

3) Delete the incomplete layers? Or do they still contain valuable information?

The layers contain the items that QRecall could capture before it was stopped. Whether that's "valuable" is entirely up to you. The only danger of incomplete layers is when recalling; if you recall an incomplete layer you may be missing items that were modified before that layer date, but were not captured.

If incomplete layers bother you, simply merge the incomplete layers with the next complete layer in the archive. You'll discard any intermediate versions of items that were captured in those incomplete layers, but at least you won't have any incomplete layers in the archive.

Prion wrote:There may be a simple thing causing all this but I am at a loss.

I wouldn't bet on "simple."

There are a bewildering number of issues that can cause an archive to become unusable. The most common is a corrupted volume structure. This can cause the operating system to report I/O errors, mysterious "Cannot convert FSRef to Path", data validation errors, and items to disappear in the Finder—I'm not sure if this is what you meant when you wrote "Archive disappeared."

Once QRecall detects any inconsistency in an archive it will refuse to use it until it can be repaired. So the general formula to dealing with these kinds of problems is:

(1) Repair the volume that contains the QRecall archive using Disk Utility.
(2) Repair the QRecall archive using the File > Repair... command.

The original/root cause of the problem can probably be found in the QRecall log. It will most likely be the first error or warning that immediately precedes the spat of problems you're having now. If you get stuck trying to figure it out, send a diagnostic report (Help > Send Report).

Prion,

I received your diagnostic report. The "Unable to read extended attribute data" message occurs when QRecall tries to get the attributes of your remote volume—the volume containing the archive. QRecall uses some attributes of the archive volume to determine compatibility with advanced features.

I've never seen this happen before. I suspect that it's an idiosyncrasy of the TimeCapsule.

Anyway, it's harmless. The extended attributes of your archive volume's root directory aren't important to QRecall.

Prion wrote:ah, bummer, although I half expected that. Does the same hold true for the TimeCapsule itself? Never tried that one because it keeps being used for TimeMachine backups of the family's laptop.

To the best of my knowledge, all of the volumes in and connected to a TimeCapsule are accessed the same way.

Regarding the extended attributes I must admit I never noticed the little details slider in the upper right corner of the log window. It helps a lot to pull it all the way to the right

It reads:

xattr:com.apple.system.Security
errno: 1

Errno 1 is "Operation not permitted" which is pretty generic. Since QRecall is trying to read a security attribute, I assume it has something to do with the security services, but I have no idea what and I've never had that happen here.

If you send me a diagnostics report (Help > Send Report) I might be able to tell more. Let me know if repairing the volume makes any difference.

Prion wrote:- QRecall cannot find the archive when I try to capture to the drive connected to the TC while logged out

That's expected; OS X can't mount a network volume unless you are logged in.

A Time Machine volume appears as a networked volume. Networked volumes require a user and password to mount, which requires that either you enter the name and password via a dialog or you save the name and password on your keychain. Both of those solutions require that you be logged in.

- when logged in, the capturing takes place normally, however, the log specifies that "Unable to read extended attribute data". This does not happen when plugging the drive directly into the MBP

That message says that the operating system reported that a file or folder has extended attributes, but when QRecall tried to read them it returned an error instead. You might trying repairing the volume and see if that makes any difference.

The file that encountered the problem, the name of the extended attribute, and the error returned by the operating system are included in the details of the message. If you need help deciphering the cause or its implications, send a diagnostic report (Help > Send Report) and I'll take a look at it.

Daniel Newman wrote:Thanks for your reply. So basically I need to install QRecall on each computer then. The backups go into one archive, but segmented by different owners (i.e. each computer) and the backups instigated by each computer individually. Is this correct?

Perfect.

So there is no real network clients or cannot be instigated by one computer/server only?

Not in the current version. The current version shares archives via file sharing. There's no "backup server" in the traditional sense.

Daniel Newman wrote:So far in my testing QRecall seems to be a fantastic and flexible backup Program.
I am seriously considering to purchase and use it for backing up a small office, which includes of 2 main computers and a server.

Daniel,

You do need three keys, if you want to capture all three computers (two main and one server) to the same archive. Install one key on each computer.

Every item captured in an archive belongs an owner. An identity key identifies the owner. To see the owners in an archive, open an archive and choose View > Show Owners and Volumes.

Let's say you install three different identity keys on three computers and name them "A", "B", and "Server." When computer A adds its files to the archive, they appear in the archive under owner "A". Similarly, the same files captured using the server appear in the archive under owner "Server."

An identity key uniquely identifies an owner and is used to "tag" the ownership of every captured item. You can change the name of an identity (QRecall > Preferences > Identity Key). Its name will change in the archive, but it's still the same identity/owner and any items captured using that identity key still belong to that owner.

You only need an identity key to capture items. If the server isn't capturing its own files (just hosting the archive file and running maintenance actions), it doesn't need to have an identity key.

Tip: If you're using the Capture Assistent to set up your actions, delete the merge and compact actions created for the two networked computers. Those two only need capture actions. If they are all sharing the same archive, only one computer needs to be performing the regular maintenance (merge and capture) actions, and the one directly connected to the drive containing the archive is the most efficient.

Axel Munker wrote:I don't know if it's only me,

It's not just you...

QRecall sends signals to the operating system while it's working to keep the computer awake. This was done because a few early QRecall users were complaining that their computer was going to sleep in the middle of an action. I've never seen it happen here, and I suspect that it depends on certain combinations of OS and hardware. These "stay awake" messages have the unfortunate side effect of also waking up your monitor.

You can turn this feature off with an advanced QRecall setting. Close QRecall, open a Terminal window, and issue the following command:

defaults write com.qrecall.client QRRunningMakesSystemActive -boolean false

In version 1.2 I've put this option in the QRecall preferences window, making it much easier to set.

Charles Watts-Jones wrote:OK I understand this and even suspected it though it doesn't explain why two verifies (2 & 5 January) started at 6:55. Or does it?, it's too late here to do the maths.

The action is probably scheduled to start at 6:10 but isn't starting until 6:55. The action might be waiting for the archive or maybe the computer is asleep and that's the time you have it programmed to wake up.

By the way my own machine uses the same verify routine and has had no troubles (so far as I'm aware). Crazy world, eh?

It is. And as soon as I find a better one, I'm moving.