I keep re-reading the Rolling Merge Action page, but seem to fall short in understanding exactly how the "Followed by:" merging works and what effects my choices will have.

If I said Keep most recent 30 days and left it alone, what happens to the 1st day once I've reached 31?

If I say keep most recent 30 days, followed by 30 day layers, is that not the same as simply keeping the most recent 60 days?

If I say keep most recent 30 days, followed by 3 week layers, then:
- The 1st day gets bumped into the first week layer
- Once the first week layer has 7 "bumped" days, then, what, the first of those 7 days gets bumped to the second week layer? Or the whole first week layer gets bumped to the second week and the first week layer now contains only the next bumped day?

If I set up like this, what happens?
[list]Most recent 30 days
3 week layers
2 month layers
1 year layer

If I don't do anything beyond "Most recent 30 days," the bumped days get merged with the original?

Sorry, it seems clear until I try to think it through. Perhaps the help page could use some specific examples to help understand what the real effect of choices in the action will be?

Thanks,

Steve

Steven M. Alper wrote:I keep re-reading the Rolling Merge Action page, but seem to fall short in understanding exactly how the "Followed by:" merging works and what effects my choices will have.

Rolling Merge is really a simple idea, but it has been a struggle to explain it clearly. Let me explain the logic that QRecall uses and see if that makes it any clearer. Then I'll try to answer your other questions.

Each "tier" in the rolling merge defines a set of time periods, counting backwards from midnight today. Let's say you defined a rolling merge that has no recent layers, 5 day layers, 2 week layers, and 1 month layer. When that rolling merge is run on Friday May 4th, 2007, it would find the following time periods:



- 0 recent layers: nothing
- 5 day layers: One period each for April 29th, 30th, May 1st, 2nd, and 3rd.
- 2 week layers: One period each for the week of April 22nd and the week of April 15th.
- 1 month layer: All of the days in March, plus a second period for April 1st through the 14th; this represents the fractional month between the end of the last month layer and the beginning of the first week layer.
- Remaining layers: Everything before March 1st.

All of the layers in each time period is then merged into a single layer. If this archive only had one layer per day, then the first 5 day layers would do nothing as those days already have only one layer.
All of the layers between April 22nd-28th would merged into a single layer. (week layer)
Layers between April 15th-21st would be merged together. (week layer)
Layers between April 1st and the 14th would be merged. (partial month layer)
Layers between March 1st and the 31st would be merged together. (month layer)
Everything between the first layer and February 28th would be merged. (remaining layer)

If I said Keep most recent 30 days and left it alone, what happens to the 1st day once I've reached 31?

Assuming that you are created one layer each day, the 1st and 2nd layers would be merged together, leaving you with the 30 most recent layers.

Said more precisely, any layers captured in the past 30 days would be left alone. Any layers captured more than 30 days ago would be merged into a single layer.

If I say keep most recent 30 days, followed by 30 day layers, is that not the same as simply keeping the most recent 60 days?

If you create one layer a day, they are equivalent.

As a counter example, I have QRecall capture the source to the project files every 20 minutes while I work. At the end of the day there might be 20 new layers in the archive. After working on QRecall another 29 days I might have 600 new layers. (Yikes, that's a lot of layers!)

If the rolling merge was set to keep the most recent 30 days followed by 30 day layers, it would leave all 600 layers as they are because they were all captured in the past 30 days.
If the rolling merge was set to 0 recent days and 60 day layers, the 20-odd layers created each day would be merged into a single layer, reducing those 600 layers into 30.

If I say keep most recent 30 days, followed by 3 week layers, then:
- The 1st day gets bumped into the first week layer
- Once the first week layer has 7 "bumped" days, then, what, the first of those 7 days gets bumped to the second week layer? Or the whole first week layer gets bumped to the second week and the first week layer now contains only the next bumped day?

A complete week layer is a single layer. What happens the next day is that there is a gap -- a fractional time period -- between the most recent week and the end of the 30 day. Over the next 6 days all of the layers in that gap are merged into a single layer. At the end of the week, that single layer becomes the most recent week. At the same time, the oldest week falls off the end (rolls) into the first monthly period, or the factional portion of the month, and is absorbed there. The next day the process starts all over again.

If I set up like this, what happens?

Most recent 30 days
3 week layers
2 month layers
1 year layer

Whatever layers you capture in the past 30 days are left alone. Layers in the recent period are never merged.

All of the layers between the 30 day period and the next three full weeks will be merged into 3 or 4 layers. One for each full week, and one for any fractional week between the last week and the beginning of the 30 day period.

Moving backwards from the earliest week, QRecall finds the next two full months and merges all of those layers into 2 or 3 layers (again, two full months plus a partial month if there is one).

It then moves backward and finds all the layers captured during the previous calendar year and merges those into one layer. A second layer might be created if the earliest month layer wasn't January.

All of the layers created before Jan 1st of last year are then merged into a single layer.

If I don't do anything beyond "Most recent 30 days," the bumped days get merged with the original?

Yes. The layers in the "keep" period would be left alone and anything beyond that will fall into the single "remaining" layer.

Sorry, it seems clear until I try to think it through. Perhaps the help page could use some specific examples to help understand what the real effect of choices in the action will be?

It does seem simple -- until you start to worry about the details. Personally, I rather people not worry about the details and simply focus on goal: What resolution and duration of incremental backups do you need?

If, two months from now, it is important to be able recover both the version of a document from Thursday May 3rd and Friday May 4th, then you'd probably want daily layers going back 90 or 120 days.

On the other hand, if you only care about getting back files in the past few days as well as that vacation video you deleted last November to clear up some disk space, then keeping the past 7 days plus 12 month layers is probably more in line with your needs.

I keep struggling to find a simple way of explaining this. I'm considering adding some more detailed examples to the help file. My reluctance is that I try to keep the help short and to the point, and explanations like this get rather tedious.

Before I go into my continuing difficulties with the concept, I need to clarify something else that I think I've misunderstood:

Are Merges and Rolling Merges exclusively for manipulation of the archive itself? There's no capturing going on?

So in other words, I would likely need three actions per archive?:
1. A recurring capture
2. A recurring merge or rolling merge of the resulting archive
3. A recurring compact of the archive

Somehow I (mistakenly?) got the impression that the Rolling Merge included a capture and then the merge.

Like I said earlier: it all seems so simple and straight ahead until you really start implementing. And I'm sure it ultimately is once some kind of contextual leap is made.

Best,

Steve

[P.S. - I hate the way (at least your installed version of) JForum ignores the BB Code on first submission (actually displays the code) until you edit and resubmit.]

Steven M. Alper wrote:Are Merges and Rolling Merges exclusively for manipulation of the archive itself? There's no capturing going on?

That's correct. A merge action only combines layers that have already been captured. Captures and merges are complements of each other: A capture adds a new layer, a merge removes old layers.

So in other words, I would likely need three actions per archive?:
1. A recurring capture
2. A recurring merge or rolling merge of the resulting archive
3. A recurring compact of the archive

And if you're paranoid (like I am), you'll want to schedule a recurring verify.

Note that I'm contemplating adding an assistant to QRecall that would help you set these up the first time.

[P.S. - I hate the way (at least your installed version of) JForum ignores the BB Code on first submission (actually displays the code) until you edit and resubmit.]

So do I. I'm working to see if I can upgrade to a version that has some of these problems fixed. I find I have better luck when I uncheck the "Disable HTML in this message" option.

I've had good results with the Simple Machines Forum. (Of course, my forums have been even more low key than this one....)

Steve