QRecallDownloadIdentity KeysForumsSupport
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Groups] Back to home page 
Messages posted by: Mark Gerber
Forum Index » Profile for Mark Gerber » Messages posted by Mark Gerber
Author Message
I'm working away from my regular workplace for a few weeks and was unable to bring the external drive on which my archive is stored. Obviously I want to continue my backup process each night but would rather not create an entirely new archive (it will be a few hundred GB and I don't think I have the space for that).

Ideally, I'd like to start a new archive of the folder in which I'll be doing most of my work over the next few weeks. Can I do that and then, when I return, use Combine to merge my full archive and this folder's archive?

If that's not possible, what would you advise?


Yes, it turns out I had somehow managed to create three captures and it was easy to use Combine Items to bring them all together.
What does it mean when the Layers Browser are displayed in gray and the layer number has no color? Are those the daily backups that have not been merged?

I'm trying to recall a single file from my most recent backup (Layer 127) and everything after Layer 41 (February 11, 2014) is gray.
The folder containing the specific file I'm looking for isn't displayed in the Item Browser. In fact, the most recent Modified Date in this group of folders is February 2014.

As far as I can tell, the Shades are revealing all layers so I would expect to see files from recent updates. What am I missing here?

James Bucanek wrote:
You have a number of factors conspiring against you.
(shaking my fist at the gods, now)

Thanks, James. I was most concerned that something had gotten messed up with the upgrade and capture delay, resulting in a corrupted archive. Your detailed answer alleviates those fears. Sounds like everything is working as I should expect, given the circumstances. And I suppose I should expect the same when I eventually get that new Mini with the Mavericks, offset somewhat with much more RAM and other Faster Stuff.
A few days ago I finally upgraded my Mini (2007) to Lion (10.7.5).
Because I was in the midst of deadlines, I suspended QRecall's (1.2.3. schedule and, of course, forgot to start things up again until last evening. Since then, QRecall has been capturing for over 8-½ hours and it's only about ? done. Normally, this takes less than 2-½ hours.

I noticed this line in the log and wondered if it might be relevant:
"Filesystem change history ignored"
If it's not, do you have any idea why this particular capture might be taking so long?
It looks like it's going to take another 4-6 hours to complete and, in the meantime, my system is not as responsive as I need it to be.
If I suspend or quite QRecall now, will it start all over again? Is it better to keep it running in the background?

Reading another user's question about FileVault and their experience with an exceptionally long backup, I checked and I do not have FileVault enabled.

Thanks for your help.
Thanks for digging in to that, James.

I tried using Activity Monitor to stop the runaway process but, while QRecallHelper did disappear from Activity Monitor, the QRecall Activity window showed it was still hanging. Ultimately, I restarted the MacBook, launched QRecall, mounted the drive, and started the capture again. Seems to be going fine for now.

As far as the mount/unmount sequence, I have no idea what might be causing it. I do have a sequence of actions that run from the Mini starting at 4:00 AM, but this problem seems to occur before, during, and after that process.

I have submitted a support ticket to OtherWorld Computing and I'm hoping they might have some clue. Or is it possible something in the network would initiate the connect/reconnect sequence?

It sounds like what I need to do is set things up so the actions are being run from my wife?s account (using one ID key). So a few days ago, I did this:

- Changed the permissions on the archive so anyone can read and write to the file (which is on an external drive attached to a Mini).

- Launched QRecall from Steph?s account on the Macbook and used the Capture Assistant to set up a basic series of actions to write to the already established ?macbookpro-all users-full.quanta? file.

I think QRecall did a relatively successful capture two days ago (there are some minor files we seem to be having problems with). I left the drive mounted, hoping things would settle down to a routine. Upon checking yesterday's log (which I'll be sending to you), I see there are several entries that read, for instance, "Stopped processing folder Images". Something about not being able to read the envelop content length.

- This evening, from my account on the MacBook, I deleted all the actions in QRecall.

However, the QRecall Activity window shows a Capture in progress, stuck for the past 25+ hours at less than 13 MB captured. I tried clicking the little x in the window, but nothing changed.
I went back in to Steph's account and the Activity window was there, too. The Actions window says a Capture is Running. I clicked the x in the Activity window again--still nothing.

How can I stop this process?
Have I hopelessly messed things up and do I need to start over and, if so, will I be able to use the existing archive so we don't have to go through building another 65+ GB file?

I'm running a copy of QRecall on my wife's MacBook Pro.
I logged in to my account on the MacBook and, using the Capture Assistant, set up actions to capture the Startup Volume to an archive on an external drive attached to a Mini.
The initial backup was successful, but there have been no subsequent captures or merges.
The QRecall Activity window on the Macbook has four actions that are waiting.
The Actions window has red "!" in front of each action with the note, "Waiting for action to finish."

I'm not sure, because I don't spend much time on that computer, but I think my account on the MacBook was logged out at some point. So I assume the actions could not proceed.

Is there a way to kick QRecall past this waiting state?

Should I set it up again through my wife's account, which is always logged in, and, if so, will I have to start over with the first capture or can QRecall recognize the first one on the Mini?

If the MacBook is not connected to the Mini through the network, will it automatically do this or should I make sure of that before retiring for the night?



If you haven't quite grasped the QRecall paradigm and vocabulary, I would suggest taking a look at the screencasts on the home page under "Key Features." They helped me quite a bit in getting my head around some of the ideas.

Thanks for the screen shot of your AM window. Mine did not look like that at all during that extended verify (finished after over 28 hours). As I mentioned earlier, my peak throughput (?) was never more than around 1.5 KB/sec. More often than not, the graph showed a lone green spike followed by a long, low flat line. I envy your high green line with the big numbers.

So I don't know if this is a problem with some part of the network (which, as far as I know, could involve the cables, internal routers, or even how many people are on this party line called cable broadband), the NAS drive itself, or a glitch in the data that's being captured. My inclination is to think it's something to do with the network. I've sent another report to you and perhaps you can confirm that the data is not the problem.

Looking at the log after the verify finished, it looks like attempts made to capture data from the Mini to the NAS drive have not been successful.

This has lead me to re-think the idea of using a NAS drive to backup our three computers. The bulk of the data that needs to be backed up comes from my computer (the Mini) and, since a data transfer over a network is inherently slower than to an external hard drive, I'm inclined at this point to exchange the NAS drive for an external drive (still set up as a RAID 1) on the Mini. For the large, first-time captures made on the other computers I'll just hook the drive up to them. The rest of the time, I'll set up their copies of QRecall to mount the Mini and capture to that external RAID. The NAS drive may just be adding a layer of complexity that I don't need.

Hope that makes sense--sort of just trying to figure this out by typing out loud.

James Bucanek wrote:As long as the numbers are changing, then it's probably working. But 10 hours seems too long.
Sometimes it's only the timer that changes. For instance, the quanta number has been stuck at 440.506 for at least 15 minutes. But after a while, it will suddenly get over that hump and everything seems to fly. So far, I have no idea how long "a while" lasts.

I don't know what your network speeds are, but it should certainly be able to verify the archive faster than the 3 hours is took to write it. If the verify is slow or getting stuck, then something (network, the NAS drive, ?) is stalling or retrying.
I'll get back to the OWC tech support and see if they can help me determine if that's the case.

When it's done (or even before if it's going to take forever) send another diagnostic report. Or look in the log for any data retries reported by the verify.
At this point, some 16:35 into the process I'm inclined to let it finish. I estimate the progress bar has only traversed 2/3 the distance, though. When it does finish, I'll send a new report.

You can also run Activity Monitor and see if the QRecallHelper process is working and what your network activity is. While a verify is running, the I/O should be pretty much saturated during the initial "Verifying archive data" phase.
Unfortunately, I have little understanding of how to interpret the information Activity Monitor presents. The Network graph shows occasional spikes up to around 1.56KB/sec with nothing else but the verify process running--but it will jump to 65KB/sec when grabbing a page from the web. QRecallHelper is running: the numbers under the CPU column bounce from 0.0 to 0.1, threads seem to stay around 10, and it's using 1.39GB of virtual memory.
James Bucanek wrote:The server thinks there's still some phantom client out there that has the archive file open. The easiest way I've found to fix this is to stop and restart the file server (System Preference > Sharing, turn file sharing off then on).

Now about that NAS drive. There some I/O errors early on in the log, which is bad and QRecall can't do much about a drive that it can't read or write to. Most of the errors, however, are data errors. The suspicious thing is that they all occur around the point where the archive has grown to about 4GB.
When you say "early on" are you referring to log entries prior to this past Sunday (7 Dec)? If so, those backups were going to the other computer (a G3). I didn't get the NAS drive until this past weekend.
Is it possible to tell if those errors are related to problems transferring the data or problems writing to the drive?

First make sure the NAS volume is formatted as Mac OS Extended, then make sure this NAS device doesn't have problems with files over 4GB. If you have a good archive on another volume that's larger than 5GB or so, verify it, copy it to the NAS, then verify it again there. If the copy or verify fails, the NAS probably can't handle file greater than 4GB.
Yesterday afternoon I spoke with the drive's tech support; the drive is formatted as Extended 3 and shouldn't have problems with files greater than 4 GB. He did echo your suggestion that I test it by copying a large file there and back again.
So, here's what I've done so far:

- I successfully copied the 19 GB archive on the G3 (mini documents backup) to the NAS drive. It took about 3 hours.
- Opened the archive and ran "Verify".
- After about 1-1/2 hours and with the status window still showing the same numbers for the past hour (about 70.000 quanta, , 2.8 GB, and 15 folders) I felt I should force quit (I know, patience!).
- Restarted the computer, launched QRecall, opened that archive again.
- I ran Verify again and this time let it run through the night.
- Now, nearly 10 hours into the process, the numbers are at 347,765 quanta, 9.92 GB, 2,280 folders, and 61,890 files. It's still cranking. I suppose that's good.
- But! I forgot to start and restart the G3's file server as you suggested above. And I did not use Verify on the original archive on the G3 before copying it to the NAS drive.

Would any of these, along with the unwise force quit I did last night, compromise the Verify process that's running?
Is it unusual for Verify to take this long over a network?

- I am now using a copy of QRecall on the G3 to verify the original archive.
- Verify finished after only 26 minutes.
Until today I have had success capturing data from my documents folder to an archive on another computer. Since this morning I've been having nothing but trouble capturing that same folder to an archive on a new NAS drive.

I went through the wizard last night to set up captures using both the All Users and my Home folder options.
- The log for the All Users archive shows an "archive I/O error" after running for only few minutes.
- The Home folder log seems to indicate problems capturing a specific file. I've run that action twice since then and each of those times there was a problem capturing the same file or the folder it's in. That file is an internal DEVONthink database backup, but DtPro wasn't running at the time.

Using auto repair on the Home folder archive reduces the size of the archive from about 3.5GB to a few megabytes.

Does the fact that these files have been captured successfully to the first archive on the other computer indicate a problem with the drive or is there a new problem with these files?

If the problem lies with the files and not the drive, should I simply exclude these files from the capture or should I be looking for some underlying problem?

On another note, I'm afraid I have a bad habit of double-clicking a folder listed in an archive to open it. Of course, this begins the Recall process which I try to stop before that process is finished. Do I have to locate and delete this partially recalled folder or is it automatically deleted when the process is stopped?

James Bucanek wrote:Also correct. This was discussed some time back in the "QRecall and CoreData" thread.
Oops. Now I wonder if what you wrote is what I'm remembering. Should have searched the forums a little better than I did.

To specifically address the issue of databases, I'm planning a new filter option that will ignore a folder full of files if any of those files are currently open for modification (write access). The capture would always be "safe" in that the capture would only occur if all of the files are closed.
That would be a welcome solution. But until then it sounds as if I will have to exclude any file that might have an open database in a folder I intend to backup (I work at home, my hours aren't regular, and I don't know if I can be disciplined enough to quit those programs that might fall into this category).

Is there someplace a list of those applications using CoreData? Or am I perhaps being too paranoid about this problem?
Some of the programs I use recommend their databases not be backed up while the program is running. Specifically, I'm thinking of DEVONthink Pro, PowerMail, and SOHO Organizer (which uses OpenBase).

Both DtPro and Organizer allow one to set the number of backups the application itself will keep, but I want to have another backup, preferably through QRecall, on another disk. And, of course, I'd like to capture these files a few times during the day. It's my impression the potential for damage is due to the fact that these databases needed to be closed before copying, otherwise an incomplete file will be written.

For this purpose, does QRecall do anything different in capturing quanta so I wouldn't have to be concerned about quitting the program to ensure a complete, undamaged capture?

By the way, I just found the screencasts on your home page. They are well done and present the information very clearly.
I look forward to seeing more, in particular, one that clarifies the rolling merge options.

Forum Index » Profile for Mark Gerber » Messages posted by Mark Gerber
Go to:   
Powered by JForum 2.1.8 © JForum Team