<![CDATA[QRecall Community Forum - Latest posts for "Mostly Harmless"]]> https://forums.qrecall.com/posts/listByUser/894 JForum - http://www.jforum.net [Suggestions and Feedback] Re:1:1 copies, source data validation...
Thanks for taking the time to work through my catalogue of questions and suggestions

I have this crazy dream, from time-to-time, of writing a file system plug-in that would mount a QRecall archive as though it was a volume containing the original files (read-only, of course)

There may be something that could serve as a basis of this at google code.
One implementation makes .flac files appear as .wav. And this is read & write.

Keep reading; you might change your mind.

There are many reasons why I don`t want any archiving in open or proprietary standard.
A few of these:
I want to be independent of a particular backup solution, comparing backup and original with other software, the more complex the storage mechanism, the more severe damage might be the consequence if there is a bug, backups surely require more time using packeging (be it for compression or for your method to make the most use of storage space), manual checks of individual files for piece of mind, a damaged backup archive always endangers more than one file.
QRecall`s storage mechanism is more complex than 1:1 copies as it does store fragments of files and relies on it`s database what piece of the puzzle belongs where to re-assemble the pieces in case they should be restored.
I think the benefit is supposed to be able to backup and store a total amount of data that is virtually bigger than what would be possible to store within the particular storage device`s capacity even if compression would be used.
I admit the potential is great if you are using versioned backups or archiving.

I've considered implementing a "compare" feature that would compare the file information in a QRecall archive with what's on the disk. Surprisingly (up to now) no one has requested this feature.

I am not surprised. I wasn`t interested in this prior to 3-4 days ago when I read why this is an issue at taobackup.com
This is clearly something that is not totally obvious.
Nevertheless I agree with the author of taobackup that this is an important issue.

The problem with a compare function as an integrity tool that that files change all the time. ... a compare feature can't tell you if one of your source files is damaged, only that it's different.
...No software ... can tell you if a document has become damaged. Comparing the files on your disk with the copies in your backup only tell you what's changed, not what shouldn't have changed but did.

The modification date is changed by the operating system whenever there is write access.
If the modification date does not indicate write access since a hash signature was generated but the file has changed, then it is corrupted.

By keeping efficient multi-generational copies of files, should you find a corrupted copy of a source file you can go back to the archive, go through its history, and retrieve a good copy of it before it became damaged.

The problem is that it is very likely that you won`t notice the difference.
While you can very often *see* the difference in a picture file, in case of software for instance that you bought as download and not on a DVD-Rom you can`t tell it was damaged. The software may now crash often but you will blame the developer.
There are more examples. Chances are that you will not discover it.

A Verify action can be scheduled to run on whatever schedule you choose.

So far for the destination only.

A transient permission or read error with a single file will be logged, but won't stop a capture. Errors with the archive itself will immediately stop a capture, in order to prevent further damage to the archive.

That`s what I aim at when I want 1:1 copies:
If there is just a small defect in the archive, maybe due to a bug or just a powercut when writing the backup, a large number of backed up files may not be possible to restore any longer (if data fragments where damaged which are required for re-assembly of many files (versions)).
That`s why my approach is to keep it as simple as possible. But I also think that verification is necessary (some backup solutions for the Mac with 4 star ratings don`t even check the destination after writing. Funny: the developer almost seemed to present this as a feature because it speeds up the backup process. well, I hope he has a harddrive crash soon and learns that his complete backup is useless ).
So there are many simple 1:1 copy tools (free) and commercial solutions. But some have other shortcomings. Command line tools are not for me.

QRecall has "event" schedules that can start a capture when the volume containing the items to capture is mounted, or when the volume containing the archive is mounted.

Very flexible

If the application write a monolithic file using the common "safe save" technique, QRecall will capture the old copy of the document intact. If not, then the captured copy may be inconsistent and you'll have to perform another capture of the document while it isn't being modified before you have saved a valid copy. All file copy and backup program suffer from the same problem, even Apple's Time Machine.

Is there no way for a backup application to check if a particular file is currently being saved?

In the activity window, you have two choices. Stop and Reschedule cancels the current action and schedules it to run again at some future date. Or, you can pause a running action for varying amounts of time from between 5 minutes and 4 hours. The action will automatically resume after that time, or you can manually resume it at any time.

This is very thoughtful. The perfect option for any occasion


There are many things that speak for QRecall. The lack of a more simple 1:1 copy sort of spoils it for me. An archive or versioned backup is more or less an afterthought for me while I consider simple 1:1 copies to be important.
]]> https://forums.qrecall.com/posts/preList/202/878 https://forums.qrecall.com/posts/preList/202/878 GMT [Suggestions and Feedback] 1:1 copies, source data validation...
I just discovered QRecall on versiontracker after I had looked into everything listed at macupdate.com.
It`s clear that data integrity is important to you - unlike to some other developers of Mac backup software.
That makes QRecall an interesting candidate although here too I see a few clouds on the horizon.

What I really want is 1:1 copies of source data. No compression. I want to be able to access the data with finder and other programs for testing and restoring.
Would this be optionally possible with QRecall (planned) or would you consider to add this feature?
I understand this is not the way QRecall is designed to work on default but of course it would be pretty easy to implement compared to the more complex system used by default (which safes disk space).
I would probably use the more complex method for daily backups. But at least once a week I want a 1:1 backup without any tricks - just to play it safe.
If you will not offer 1:1 copies you don`t have to read the rest of my post because I am afraid this is a must for me.

I would also like to be able to turn of archiving. I will probably use it but have not made up my mind, yet. Should be an option I think.


Verifying data:
1.
From your site I learned that backed up files are verified.
Usually I would ask if this is done by comparing the backed up files bit-by-bit to the original or with hash files.
I noticed however that QRecall doesn`t necessarily store complete copies of files if they are just new versions of files that already exist.
As I wrote I am looking for a solution that does 1:1 copies and I want these to by verified either by bit-by-bit comparision to the original or by using hashes derived from the original.
To have this with QRecall would first require that you use 1:1 copies as an option.
2.
Validation of source data:
Actually this is a *very* important feature though it`s not implemented in any backup solution for the Mac that I know of (and I am pretty sure I know about pretty much all products available).
I know that at this point of time I may have to to it with an additional program but I am hoping to see this integrated in a professional backup solution for the Mac in the future.
So I am now proposing this to you:
By creating and *saving* hashes for each file that is going to be backed up from the source harddrive, both the original source data and the backup copies can be verified days or weeks later.
Why?
The original source data can be corrupted. This can happen for example by a crash/powercut while writing to disk, by software bugs (think of an encrypted sparse image containing a complete user home folder protected by FileVault), harddrive defect (pretty likely these days), malware.
In most cases this data corruption will not be noticed for quite some time.
Until then the defect files may already have been replacing all previous copies on daily, weekly and monthly backup drives.
So chances are that you have bought 3 external harddrives for redundancy, maybe you keep one offsite. You have carefully chosen and paid for a professional software solution.
You do all your homework but in the end you still lose data.
Not because it`s not backed up but because both the source and all backups are damaged and it wasn`t noticed.
This is worth considering, isn`t it?

http://www.taobackup.com/integrity.html
"...You will not achieve enlightenment until you control the integrity of your data, for a copy is useless if the original is corrupt."

http://www.taobackup.com/integrity_info.html
"The greatest danger to your data is not catastrophic failure, but subtle damage that goes undetected. The corruption of several files on your disk may cause great damage to your business in the long term, but go entirely unnoticed in the short term. If it does go undetected, the corrupt files will flow through your backups until there are no uncorrupted copies left."

Due to the processing power to calculate hashes and the drive space needed to store the hash data I would probably want to use this feature only on the most important data (stuff I create myself). If would be nice to be able to browse the home directory and activate a checkbox for the feature for any folder.

A scheduled validation of the source data would be what I am dreaming of.


The rest:

I want to be actively informed of any errors (with a pop-up alert panel, some blinking symbol in the menu bar, alarm sound or a clearly visible statement in a summary at the end of a backup-job).
One competitor`s solution only writes an error to the log if the backup-job was started by scheduled (if the backup is started manually it does inform the user).
What I don`t want is the backup-job to stop.
For example:
Maybe I start a backup before I leave. When I come back I discover that the backup stopped because the user(-account) had no read permission for one folder or file. All other files could have been backed up but the program stopped the whole process when the error occurred. Not very smart.

Start a backup-job when a particular volume is mounted.

What happens if a file is saved while QRecall is backing up this particular file?

At versiontracker the info says "beta".
Is it a beta or a finished product?

I would like to be able to pause a backup-process (if I need the resources to do something with the computer). It would be nice if the process could auto-start again after while, if the computer is idle (screensaver on?), in case I would forget to continue the backup (by pressing pause again before I leave the computer).
]]> https://forums.qrecall.com/posts/preList/202/875 https://forums.qrecall.com/posts/preList/202/875 GMT