Hi James,
on a few of my archives I set the preference "Keep deleted items for at least" to a very high value (200 months) so to force Qrecall to keep all deleted files and don't erase them during a compact activity.

I launched a Compact command on a couple of that archives and found on Log that:
1. there are a lot of entries "Erased deleted items from layer xx"
2. Qrecall was able to free and recover some space (a few hundreds MB out of 20-30 GB)

How is it possible to free and recover space if no files are to be deleted?
Is there something I'm missing about compacting?

By the way, besides Compacting archives, do you suggest any other maintenance activity on a disk storing Qrecall archives? Will defrag be useless or even dangerous?

Thanks,
Pierpaolo

pirem71 wrote:on a few of my archives I set the preference "Keep deleted items for at least" to a very high value (200 months) so to force Qrecall to keep all deleted files and don't erase them during a compact activity.

Hmm, maybe I should add a "forever" to that option.

I launched a Compact command on a couple of that archives and found on Log that:
1. there are a lot of entries "Erased deleted items from layer xx"
2. Qrecall was able to free and recover some space (a few hundreds MB out of 20-30 GB)

How is it possible to free and recover space if no files are to be deleted?

The "Erased deleted items from layer ..." message is surprising. This should only get logged with the calculated save-until date (midnight of today - keep deleted items duration setting) is later than the capture date of the layer. I would suggest double checking the settings for that archive. If the setting really is 200 months, please send a diagnostic report and I'll look into it.

Free space in the archive is also caused by merge actions.

By the way, besides Compacting archives, do you suggest any other maintenance activity on a disk storing Qrecall archives?

I consider the occasional verify to be essential. Compacting is optional (unless you run out of disk space), but periodically desirable for performance reasons.

Will defrag be useless or even dangerous?

Defrag won't help much. It will theoretically help the verify action (which reads the archive from beginning to end), but unless the archive is badly fragmented (which is unlikely) I doubt the difference would be enough to measure. A QRecall archive is its own kind of filesystem, which the compact action optimizes and defragments. Having that filesystem hosted by another (possibly fragmented) filesystem won't make much difference.

As for the danger, older defragmentation applications could cause irreparable harm if they were interrupted while defragmenting the volume. Modern ones, however, go to great lengths to protect the volume from being corrupted if the defragmentation process is interrupted. So as long as you're using a recent version of something like iDefrag, I think it's perfectly safe to defragment your archive's volume. I admit to having done it several times myself.

Hi James,
thanks for the prompt reply (no emergency here so take your time and enjoy the week-end; next time I'll avoid posting on Sunday ).

Back to the issue:
I'm sure my archives are set with the (admittedly) conservative "200 months" preference so I'll send you a report.

As for the recovered space I think I never used Merge command/action on that archives (in which I want to preserve everything has been captured) but not 110% sure.
On the other hand I used for sure the Archive>Delete Items... command; maybe the files are deleted but the space is reclaimed only during subsequent compacting.
Could that be the reason for the "Erased deleted items from layer ..." messages?

Thanks again for the outstanding support,
Pierpaolo

pirem71 wrote:As for the recovered space I think I never used Merge command/action on that archives (in which I want to preserve everything has been captured) but not 110% sure.

If you never use the merge command, then it doesn't matter what the "keep deleted items" setting is set to.

Deleted items are created during a merge action when a later layer does not contain an item that previously existed in an earlier layer. In this situation, the merge would normally purge the older items from the archive, leaving no trace of the item in the archive. The "Keep deleted items" option overrides this and preserves the last captured item as a special "deleted" item in the merged layer. But if you never merge layers, then the archive will have every item ever captured and can't have any deleted items.

On the other hand I used for sure the Archive>Delete Items... command; maybe the files are deleted but the space is reclaimed only during subsequent compacting.

Space for any deleted data (items or layers) may be reused in subsequent capture actions and is always recovered by the compact command. Reusing empty space during a capture is never a perfect fit, so there are always scores of small empty "holes" in the archive. The compact action moves records around so there are no empty spaces, and then truncates the size the archive to release that unused space back to the filesystem.

Could that be the reason for the "Erased deleted items from layer ..." messages?

No, but if you've only deleted items and have never merged, then that's kind of a special situation that I'll have to look into.

James,
thanks for the clear explanation; I realized that maybe I misunderstood the concept of "deleted files".

If I delete a file in the filesystem between capture N and capture N+1, the file will be shown (from capture N+1 onward) with red stripes and only if I choose to display deleted items.
That is a "deleted file" in my understanding and it is not created by any merging actions but simply derived from a change in filesystem.

According to my understanding, such a file (after a grace period set in archive preference, in my case 200 months) will be completely erased in any case by a compacting action/command even if I don't merge any of the archive layers.
Based on your reply it seems that, if I don't merge layers N and N+1, the files that I deleted in finder between the captures remains forever in the archive (as they belong to layer N that is still there).

The confusion rises from the term deleted used in two different meaning: what will be erased from a compacting action are the "unallocated" items (the layer which they belong to doesn't exist any longer) while "deleted" items from file system are preserved.

May I suggest to use two different terms for the two file states? The Help is not so clear as well about "deleted items".

I think the confusion is due to how the "View deleted items" view option works, which is different than the "Keep deleted items" archive settings. I'll try to clarify using the following example:

(file F created)
Layer 1: captured a folder containing file F.
(file F deleted)
Later 2: recaptured the same folder, this time there is no file F.

The archive now has two layers. Layer 1 contains file F, layer 2 doesn't. File F is now a "deleted item".

Opening and viewing this archive (with all layers shown) will not display file F, because in layer 2 file had been deleted. If you then turn on the "Show deleted items" view option, file F will appear as a deleted item (hash background) because it previously existed in an earlier layer (1) but doesn't exists in the lastest layer (2).

At this point the file still exists as a regular file in layer 1.

Now, merge the two layers with the "Keep deleted items" setting turned off. When layers are merged only the most recent version of each item retained and files that have been deleted are removed from the archive. After the merge there is only one layer, and there is no file F. If you turn on the "Show deleted items" view option, you still won't see file F because it isn't there.

If the "Keep deleted items" setting had, instead, been set prior two merging the two layers then the file wouldn't have been removed from the archive. Assuming the latest version was within the "Keep" time period, instead of expunging the file from the archive QRecall would preserve the latest version in a special file record marked as having been deleted.

In the second scenario, if you once again open the archive you don't see file F. Even though the file is stored in the layer, it isn't shown because it's a "deleted" file. If you then turn on the "Show deleted items" view option, the file once again appears as a deleted item in the browser.

So the "Show deleted items" view feature will show items captured in earlier layers that have subsequently been deleted. It will also show the special "deleted" items preserved in a layer via the "Keep deleted items" setting.

The deleted items perserved in a merged layer are finally deleted for good under two circumstances. If the layer is merged again, and this time the deleted items it contains are older than their keep-until date, the items are deleted just as they would if the "Keep deleted items until" setting was off. The compact action also sweeps merged layers looking for preserved items that are now too old to keep.

Now, back to your questions:

That is a "deleted file" in my understanding and it is not created by any merging actions but simply derived from a change in filesystem.

Correct.

According to my understanding, such a file (after a grace period set in archive preference, in my case 200 months) will be completely erased in any case by a compacting action/command even if I don't merge any of the archive layers.
Based on your reply it seems that, if I don't merge layers N and N+1, the files that I deleted in finder between the captures remains forever in the archive (as they belong to layer N that is still there).

That's correct. A "deleted item" mean "doesn't exist in a recent layer, but may still exist in a previous layer."

The confusion rises from the term deleted used in two different meaning: what will be erased from a compacting action are the "unallocated" items (the layer which they belong to doesn't exist any longer) while "deleted" items from file system are preserved.

It shouldn't cause any confusion if you remember that the "Keep deleted items for at least" setting is not a "keep deleted items exactly XXX days" setting, it's a "Keep delete items for at least XX days". It doesn't set a fixed window in which all deleted items are removed from the archive. It simply creates a grace period in which items that would have normally been deleted by a merge action are, instead, not deleted. Once the item is older than that grace period, the normal item removal rules apply.

May I suggest to use two different terms for the two file states? The Help is not so clear as well about "deleted items".

I don't think there needs to be two terms, because it shouldn't matter. From the user's perspective there are only "deleted" items. How long they exist in the archive is a combination of how the layers are merged and the "keep deleted items ..." setting.

If you want to see the difference in the browser, there's an advanced setting (QRBrowserDistinguishDeletedItems) that will reveal the difference between a deleted item captured in an earlier layer and one being preserved in a merged layer. See the 1.2.0b46 release notes for an explanation and a screenshot showing the difference.

You can also disable the compact's deletion of "deleted" items by changing the QRCompactErasesExpiredDeletedItems advanced setting to false.