Yes, currently doing Spring cleaning 2020 for my computer. Way ahead of where I am for the apartment which may require organizing an archeological dig.

And this is interesting:
It looks like the decision has been taken from my hands. I still have all four disks but they now show as:
Disk 1: 11/17/19 (1.55 TB) 2,041,696 items
Disk 2: 11/17/19 (1.55 TB) 2,041,696 items
Disk 3: 11/17/19 (1.55 TB) 2,041,696 items
Disk 4: 04/24/22 (855 GB) 559,260 items (layer 1), 04/25/22 (9.5 GB) 669 items (layer 2)

Disks 1?3 indicate only one layer each (down from quite a few layers in each that I saw a couple days ago); there are two layers for Disk 4.

I'm pretty sure that my rolling merge from years ago maxed out at 2 or three years. So I think what probably happened was I must have activated the schedule without realizing it and a merge of some sort was performed after yesterday's capture. I'm interpreting this new state as there is only one layer duplicated three times.
Does that make sense given what I see now?

A quick look in one of those disks' ~/Documents folder shows I have some files going back to The Early Days so I guess everything is flattened but safe. I imagine it's what I would have done anyway but would have preferred agonizing over the choice for a few days. My mistake.

Given all this, should I delete Disks 1 and 2 as duplicates of Disk 3 and move forward with just Disks 3 and 4?

I should clarify that all four disks I'm referring to are listed under one owner/volume. I can only select one at at time and the Archive>Combine Items? menu is grayed out. Is combining still an option or is it moot since I guess I can select the root volume and search there?

When I select the Volume, I see only three layers (1: 1.55 TB; 2: 855 GB; 3: 9.5 GB). That's confusing to me given the individual disks listed. Can I actually delete Disks 1 and 2?

And then, as you wrote, my next compact will drastically reduce the size of the archive and reduce the time spent during capture/merge actions. Hope I have that right.

My QRecall archive covers four disks, each replacing the prior disk as it nearly failed or I scaled up:
Disk 1: 12/21/13 (290 GB) to 10/20/17 (925 GB)
Disk 2: 10/20/17 (925 GB)
Disk 3: 10/20/17 (925 GB) to 11/7/19 (10 GB)
Disk 4: 4/24/22 (855 GB)

The 10/20/17 backups are all the same size with the same number of items.

For some Very Bad Reasons I stopped regularly backing up my drives in late 2019. I used a rolling merge over those years that kept several days, weeks, fortnight, month, and year layers.

I want to resume my better habits and today created a new backup (Disk 4 that replaced Disk 3 back in 2019) using Capture (took about 23 hours).

My archive file is now 1.4 TB.

I?d like to clean things up and get started again with my previous setup of rolling merge actions and I?m really not sure what my options are.

If feels, FWIW, that having the archive span four files is inefficient.
Can I (or even should I) combine these individual disk backups into one?

Hoping to get some good thoughts on how best to move forward that might also result in a smaller more efficient archive.

Thanks!

Mark

I'm working away from my regular workplace for a few weeks and was unable to bring the external drive on which my archive is stored. Obviously I want to continue my backup process each night but would rather not create an entirely new archive (it will be a few hundred GB and I don't think I have the space for that).

Ideally, I'd like to start a new archive of the folder in which I'll be doing most of my work over the next few weeks. Can I do that and then, when I return, use Combine to merge my full archive and this folder's archive?

If that's not possible, what would you advise?

Thanks!
Mark

Thanks!

Yes, it turns out I had somehow managed to create three captures and it was easy to use Combine Items to bring them all together.

What does it mean when the Layers Browser are displayed in gray and the layer number has no color? Are those the daily backups that have not been merged?

I'm trying to recall a single file from my most recent backup (Layer 127) and everything after Layer 41 (February 11, 2014) is gray.
The folder containing the specific file I'm looking for isn't displayed in the Item Browser. In fact, the most recent Modified Date in this group of folders is February 2014.

As far as I can tell, the Shades are revealing all layers so I would expect to see files from recent updates. What am I missing here?

Thanks!
Mark

James Bucanek wrote:
You have a number of factors conspiring against you.

(shaking my fist at the gods, now)

Thanks, James. I was most concerned that something had gotten messed up with the upgrade and capture delay, resulting in a corrupted archive. Your detailed answer alleviates those fears. Sounds like everything is working as I should expect, given the circumstances. And I suppose I should expect the same when I eventually get that new Mini with the Mavericks, offset somewhat with much more RAM and other Faster Stuff.

A few days ago I finally upgraded my Mini (2007) to Lion (10.7.5).
Because I was in the midst of deadlines, I suspended QRecall's (1.2.3. schedule and, of course, forgot to start things up again until last evening. Since then, QRecall has been capturing for over 8-½ hours and it's only about ? done. Normally, this takes less than 2-½ hours.

I noticed this line in the log and wondered if it might be relevant:
"Filesystem change history ignored"
If it's not, do you have any idea why this particular capture might be taking so long?
It looks like it's going to take another 4-6 hours to complete and, in the meantime, my system is not as responsive as I need it to be.
If I suspend or quite QRecall now, will it start all over again? Is it better to keep it running in the background?

Reading another user's question about FileVault and their experience with an exceptionally long backup, I checked and I do not have FileVault enabled.

Thanks for your help.
Mark

Thanks for digging in to that, James.

I tried using Activity Monitor to stop the runaway process but, while QRecallHelper did disappear from Activity Monitor, the QRecall Activity window showed it was still hanging. Ultimately, I restarted the MacBook, launched QRecall, mounted the drive, and started the capture again. Seems to be going fine for now.

As far as the mount/unmount sequence, I have no idea what might be causing it. I do have a sequence of actions that run from the Mini starting at 4:00 AM, but this problem seems to occur before, during, and after that process.

I have submitted a support ticket to OtherWorld Computing and I'm hoping they might have some clue. Or is it possible something in the network would initiate the connect/reconnect sequence?

It sounds like what I need to do is set things up so the actions are being run from my wife?s account (using one ID key). So a few days ago, I did this:

- Changed the permissions on the archive so anyone can read and write to the file (which is on an external drive attached to a Mini).

- Launched QRecall from Steph?s account on the Macbook and used the Capture Assistant to set up a basic series of actions to write to the already established ?macbookpro-all users-full.quanta? file.

I think QRecall did a relatively successful capture two days ago (there are some minor files we seem to be having problems with). I left the drive mounted, hoping things would settle down to a routine. Upon checking yesterday's log (which I'll be sending to you), I see there are several entries that read, for instance, "Stopped processing folder Images". Something about not being able to read the envelop content length.

- This evening, from my account on the MacBook, I deleted all the actions in QRecall.

However, the QRecall Activity window shows a Capture in progress, stuck for the past 25+ hours at less than 13 MB captured. I tried clicking the little x in the window, but nothing changed.
I went back in to Steph's account and the Activity window was there, too. The Actions window says a Capture is Running. I clicked the x in the Activity window again--still nothing.

How can I stop this process?
Have I hopelessly messed things up and do I need to start over and, if so, will I be able to use the existing archive so we don't have to go through building another 65+ GB file?

Thanks.
Mark

I'm running a copy of QRecall on my wife's MacBook Pro.
I logged in to my account on the MacBook and, using the Capture Assistant, set up actions to capture the Startup Volume to an archive on an external drive attached to a Mini.
The initial backup was successful, but there have been no subsequent captures or merges.
The QRecall Activity window on the Macbook has four actions that are waiting.
The Actions window has red "!" in front of each action with the note, "Waiting for action to finish."

I'm not sure, because I don't spend much time on that computer, but I think my account on the MacBook was logged out at some point. So I assume the actions could not proceed.

Is there a way to kick QRecall past this waiting state?

Should I set it up again through my wife's account, which is always logged in, and, if so, will I have to start over with the first capture or can QRecall recognize the first one on the Mini?

If the MacBook is not connected to the Mini through the network, will it automatically do this or should I make sure of that before retiring for the night?

Thanks.
Mark

Fergus,

If you haven't quite grasped the QRecall paradigm and vocabulary, I would suggest taking a look at the screencasts on the home page under "Key Features." They helped me quite a bit in getting my head around some of the ideas.

Mark

Thanks for the screen shot of your AM window. Mine did not look like that at all during that extended verify (finished after over 28 hours). As I mentioned earlier, my peak throughput (?) was never more than around 1.5 KB/sec. More often than not, the graph showed a lone green spike followed by a long, low flat line. I envy your high green line with the big numbers.

So I don't know if this is a problem with some part of the network (which, as far as I know, could involve the cables, internal routers, or even how many people are on this party line called cable broadband), the NAS drive itself, or a glitch in the data that's being captured. My inclination is to think it's something to do with the network. I've sent another report to you and perhaps you can confirm that the data is not the problem.

Looking at the log after the verify finished, it looks like attempts made to capture data from the Mini to the NAS drive have not been successful.

This has lead me to re-think the idea of using a NAS drive to backup our three computers. The bulk of the data that needs to be backed up comes from my computer (the Mini) and, since a data transfer over a network is inherently slower than to an external hard drive, I'm inclined at this point to exchange the NAS drive for an external drive (still set up as a RAID 1) on the Mini. For the large, first-time captures made on the other computers I'll just hook the drive up to them. The rest of the time, I'll set up their copies of QRecall to mount the Mini and capture to that external RAID. The NAS drive may just be adding a layer of complexity that I don't need.

Hope that makes sense--sort of just trying to figure this out by typing out loud.

Mark

James Bucanek wrote:As long as the numbers are changing, then it's probably working. But 10 hours seems too long.

Sometimes it's only the timer that changes. For instance, the quanta number has been stuck at 440.506 for at least 15 minutes. But after a while, it will suddenly get over that hump and everything seems to fly. So far, I have no idea how long "a while" lasts.

I don't know what your network speeds are, but it should certainly be able to verify the archive faster than the 3 hours is took to write it. If the verify is slow or getting stuck, then something (network, the NAS drive, ?) is stalling or retrying.

I'll get back to the OWC tech support and see if they can help me determine if that's the case.

When it's done (or even before if it's going to take forever) send another diagnostic report. Or look in the log for any data retries reported by the verify.

At this point, some 16:35 into the process I'm inclined to let it finish. I estimate the progress bar has only traversed 2/3 the distance, though. When it does finish, I'll send a new report.

You can also run Activity Monitor and see if the QRecallHelper process is working and what your network activity is. While a verify is running, the I/O should be pretty much saturated during the initial "Verifying archive data" phase.

Unfortunately, I have little understanding of how to interpret the information Activity Monitor presents. The Network graph shows occasional spikes up to around 1.56KB/sec with nothing else but the verify process running--but it will jump to 65KB/sec when grabbing a page from the web. QRecallHelper is running: the numbers under the CPU column bounce from 0.0 to 0.1, threads seem to stay around 10, and it's using 1.39GB of virtual memory.

James Bucanek wrote:The server thinks there's still some phantom client out there that has the archive file open. The easiest way I've found to fix this is to stop and restart the file server (System Preference > Sharing, turn file sharing off then on).

Now about that NAS drive. There some I/O errors early on in the log, which is bad and QRecall can't do much about a drive that it can't read or write to. Most of the errors, however, are data errors. The suspicious thing is that they all occur around the point where the archive has grown to about 4GB.

When you say "early on" are you referring to log entries prior to this past Sunday (7 Dec)? If so, those backups were going to the other computer (a G3). I didn't get the NAS drive until this past weekend.
Is it possible to tell if those errors are related to problems transferring the data or problems writing to the drive?

First make sure the NAS volume is formatted as Mac OS Extended, then make sure this NAS device doesn't have problems with files over 4GB. If you have a good archive on another volume that's larger than 5GB or so, verify it, copy it to the NAS, then verify it again there. If the copy or verify fails, the NAS probably can't handle file greater than 4GB.

Yesterday afternoon I spoke with the drive's tech support; the drive is formatted as Extended 3 and shouldn't have problems with files greater than 4 GB. He did echo your suggestion that I test it by copying a large file there and back again.
So, here's what I've done so far:

- I successfully copied the 19 GB archive on the G3 (mini documents backup) to the NAS drive. It took about 3 hours.
- Opened the archive and ran "Verify".
- After about 1-1/2 hours and with the status window still showing the same numbers for the past hour (about 70.000 quanta, , 2.8 GB, and 15 folders) I felt I should force quit (I know, patience!).
- Restarted the computer, launched QRecall, opened that archive again.
- I ran Verify again and this time let it run through the night.
- Now, nearly 10 hours into the process, the numbers are at 347,765 quanta, 9.92 GB, 2,280 folders, and 61,890 files. It's still cranking. I suppose that's good.
- But! I forgot to start and restart the G3's file server as you suggested above. And I did not use Verify on the original archive on the G3 before copying it to the NAS drive.

Would any of these, along with the unwise force quit I did last night, compromise the Verify process that's running?
Is it unusual for Verify to take this long over a network?

- I am now using a copy of QRecall on the G3 to verify the original archive.
- Verify finished after only 26 minutes.

Until today I have had success capturing data from my documents folder to an archive on another computer. Since this morning I've been having nothing but trouble capturing that same folder to an archive on a new NAS drive.

I went through the wizard last night to set up captures using both the All Users and my Home folder options.
- The log for the All Users archive shows an "archive I/O error" after running for only few minutes.
- The Home folder log seems to indicate problems capturing a specific file. I've run that action twice since then and each of those times there was a problem capturing the same file or the folder it's in. That file is an internal DEVONthink database backup, but DtPro wasn't running at the time.

Using auto repair on the Home folder archive reduces the size of the archive from about 3.5GB to a few megabytes.

Does the fact that these files have been captured successfully to the first archive on the other computer indicate a problem with the drive or is there a new problem with these files?

If the problem lies with the files and not the drive, should I simply exclude these files from the capture or should I be looking for some underlying problem?

On another note, I'm afraid I have a bad habit of double-clicking a folder listed in an archive to open it. Of course, this begins the Recall process which I try to stop before that process is finished. Do I have to locate and delete this partially recalled folder or is it automatically deleted when the process is stopped?

Thanks.
Mark