I would like to suggest:

a. extending the Finder Contextual menu to enable EXCLUDING a file from capture (add to the Filter exclude list)

b. extending the filters to support Color Label exclusion (exclude stuff with label "brown")

Both are useful for those files you do not want backed up for some reason. For example, recently VMWare had a mac beta that I tried and the VMs are in my home but I don't want them 2+GB files backed up, so I can just label the folder "brown" and voila, excluded from capture...


c. extending the filters to support some form of name/attribute filter, such as name contains ".cache"

This is most useful to avoid capturing caches QRecall does not (yet?) know it's cache (probably because Apple conventions are not followed).

Fred

I have plans to eventually extent both filtering and capture item selection to include a vast array of rules, probably based on Spotlight queries.

This is fine and is c.

My other suggestion above is to allow for easier more individually selected files. Sure, functionally, launching QRecall, editing the action, opening the filter section and entering a rule that prevents "DontBackupMe.doc" to backed up is equivalent to the finder CM. It's just a lot slower and error prone, as the rule would fire for all such documents (regardless of path) whereas the CM would only remove one.

In the same vein, a "skip today" and "skip forever" button pair in the activity would be nice. If you see QRecall starting to backup a huge file you forgot to skip, you can have it done from there. The "just now" option is useful for laptops. Yes you want the backup to complete before leaving but you can tolerate that the latest Leopard preview disk of 6+ GB is not backed up today cos you're in a hurry and you can always download it later.

The problem with rule based filters is that it's hard to visualize from a set of complex rules with files will be included and which will not be. In particular, you always find a file that it's not taken care of the rule because you forgot you had it (or, for caches and the like, you did not know you have).

Fred

Frederic Thomas wrote:In the same vein, a "skip today" and "skip forever" button pair in the activity would be nice. If you see QRecall starting to backup a huge file you forgot to skip, you can have it done from there. The "just now" option is useful for laptops. Yes you want the backup to complete before leaving but you can tolerate that the latest Leopard preview disk of 6+ GB is not backed up today cos you're in a hurry and you can always download it later.

That's an interesting suggestion. That would be some logical functionality to add to the status window if it showed pending actions.

You're going to great length in the software to compress data, but one of the major causes of running out of disk space in my experience has been bad file filters.

My "skip" buttons were for the current file being backed up, so you can skip "leopard_client_9a466.dmg" when you see it go through. You just saved 6.1 GB...

I think I understand from your reply you were thinking next "capture", since you were tying to the display of the next action.

I have some issues/suggestions re the Exclusions...

1) If I read the Help file correctly, when I exclude something, and then restore a directory, anything excluded would be deleted. I would like to be able to exclude files or folders in a way that they are totally ignored - neither archived nor restored. An example here would be podcasts. They exist in my iTunes folder hierarchy, and at times they take up a lot of space, so I don't want to back them up - I can always re-download them if I need to. But if I excluded them, and then had to restore the containing iTunes folder, they would be deleted.

2) Because of this, to "exclude" certain directories/files, the only way I can see at the moment is to explicitly include all files/folders other than those to be "ignored". So, in the iTunes case, I would have to include, say, every Artist folder at the same level of the hierarchy of the Podcasts folder - that's a lot of work, and it also means that every new folder that gets created would have to be explicitly added (and may not be done so in time for a critical backup). It would be better to have an overall include, excluding ("ignoring") some of the folders/files underneath a certain level.

3) My home directory is around 60GB, but I only really need to backup about 12GB of this. So, I'm forced to use the clumsy "include just these folders" approach I've suggested above. The final problem here is that I don't think I can inlcude hidden files/folders (say, a .svn directory in home) in the current system.

I'm still just trialling the app, so I may have missed something, but so far it seems there needs to be a better way to do includes/excludes.

I agree for 1). Erasing stuff when restoring is a bad idea if it is true. Personally, save for a "restore HD" type restore, I would never restore stuff on top on itself.

For 2 and 3, the label idea (f.e.) is nice. Just set the Podcast directory in brown (say), and it's not backed up.

Stephen Murphy wrote:1) If I read the Help file correctly, when I exclude something, and then restore a directory, anything excluded would be deleted. I would like to be able to exclude files or folders in a way that they are totally ignored - neither archived nor restored. An example here would be podcasts. They exist in my iTunes folder hierarchy, and at times they take up a lot of space, so I don't want to back them up - I can always re-download them if I need to. But if I excluded them, and then had to restore the containing iTunes folder, they would be deleted.

You read it correctly. This is an excellent suggestion. I'm working on some new logic to deal with deleted files. It might be possible to extend this to handle excluded items in a special way too.

2) Because of this, to "exclude" certain directories/files, the only way I can see at the moment is to explicitly include all files/folders other than those to be "ignored". So, in the iTunes case, I would have to include, say, every Artist folder at the same level of the hierarchy of the Podcasts folder - that's a lot of work, and it also means that every new folder that gets created would have to be explicitly added (and may not be done so in time for a critical backup). It would be better to have an overall include, excluding ("ignoring") some of the folders/files underneath a certain level.

Or, you can simply capture the entire iTunes folder (excluding Podcasts) and should you ever need to restore it, select all of the iTunes subfolders in the archive and choose Restore. Each individual subfolder will be restored. Since you didn't capture the Podcast folder it isn't in that list and would be ignored. Probably much less work than the other way.

3) My home directory is around 60GB, but I only really need to backup about 12GB of this. So, I'm forced to use the clumsy "include just these folders" approach I've suggested above. The final problem here is that I don't think I can inlcude hidden files/folders (say, a .svn directory in home) in the current system.

There are a number of ways of doing this. Since the .svn directory is a folder, you can use the Go To command in the open files dialog or the Finder (Command+Shift+G) to navigate into the hidden directory. If you can open the hidden item using the Finder or some other application (BBEdit, Hex Fiend, etc.) you can then drag the item's icon from its window bar title into the action.

If you enable hidden files in the Finder, you can drag hidden items directly.

James Bucanek wrote:Since the .svn directory is a folder, you can use the Go To command in the open files dialog or the Finder (Command+Shift+G) to navigate into the hidden directory.

.svn is a delicate example for this method as a repository checked out of SVN will have a .svn directory in EVERY subdirectory. So using GO TO and selecting each one individually is not that an attractive proposition to avoid backing them up For those, a grep pattern on the path would be the best way...

.subversion is a better example of hidden home directory to reveal using the GO TO method.

Unfortunately one is likely to want .subversion backed up (it contains your local subversion config) and maybe not the .svn directories (only contains local state versus the repository)

Just to clarify for any future reader:

# you can reveal folders in the finder using the GO TO method above

# you probably want any .subversion directory in home backed up

# you may not want all your .svn folders backed up but there's no easy way to avoid it with the current (1.0.0(45) beta) version of QRecall

# a good example of a hidden home directory you would not want backed up is .dvdcss (google it to know why).

HTH

Fred

After a little more configuration and use, an ignore option seems even more like a good idea. Although the work around you suggested of selecting specific folders to restore would certainly work, it requires a lot of selection effort when only a few folders need to be ignored.

In the iTunes case, say when a whole home directory was being backed up, and with just the Podcasts folder to be "ignored", it would be necessary to individually select for restore all folders at the same hierarchy level as "Music", and then a bunch of subfolders within Music, and then every artist folder in ~/Music/iTunes/iTunes Music except for the Podcast one.

The trouble is the risk of human error, and the amount of time and double-checking, and these are the very things that cause people *not* to back up - when it becomes (seemingly) too much trouble. Of course, if something serious was going to get deleted through error, then it should have been in the backup action to start with. But this comes down to one of those "how much work/time will you lose if you don't back up" decisions, traded off against disk-space and other constraints.

In short, I think the "ignore" type of exclusion is really important.

I think a similar issue arises with the hidden folders...the Go To works, but it could be a lot of effort with things like .svn, especially in documents that are bundles (eg, Ulysses projects), and where one would have to view the package contents to find the hidden folders. If I could easily see/ignore/exclude/include hidden folders and sub-folders without having to use Finder tricks it would make me feel more comfortable knowing exactly what I am backing up.

I think we should always know exactly what is being backed up, and will be restored, and both these issues affect that.

Stephen,

Thanks for the great feedback and suggestions. However, some of these ideas are at odds with QRecall's basic design and philosophy.

QRecall was designed to capture everything that's important, quickly and efficiently, and allow you either recover earlier versions or restore any item to the exact condition it was in when captured.

The filtering mechanism was designed solely to avoid capturing superfluous or redundant files (cache, spotlight indexes, backup copies, etc.). It was not designed to micro-manage an archive, arbitrarily capturing this file but not that one.

Excluding important items from a capture creates a confusing middle ground that I'm not comfortable with. In your iTunes example, podcasts are excluded but you clearly don't want them erased. My question is "Which is it? Are the files important or not?" It would make sense to exclude them if you could easily re-download the lost podcasts. It would not make sense if you could not.

Taken to it's logical end, not capturing items and then not erasing those items during a restore would result in a mish-mash of interleaved versions. Simple examples would be backup files that are newer than the working version. Replacing some files but not others in an iTunes library will definitely cause iTunes to become confused. Or let's say you decided to reduce the size of your archive by excluding all foreign language resources from application bundles. Restoring an application package without deleting those excluded items would result in a Frankenstein mixture of old and new resource files. This could cause application instability or worse. If you did this to the operating system, the results would be disastrous.

Being be bit of a propeller-head myself, I'm all for giving sharp knives to those who want them. I am considering adding an "Overlay" command. This would be a restore that doesn't delete any existing files that do not appear in the archive; it would only recall deleted files and replace newer files with the version from the archive.

Combined with arbitrary rules for excluding items, you could (at your own risk) overlay a set of captured items on top of an existing set of files. (Note that you can accomplish this right now by recalling the items to a separate location, then merging the two directories with a little command-line work.)

I think we should always know exactly what is being backed up, and will be restored, and both these issues affect that.

I agree—which is exactly why I don't like this feature.