Message |
|
To followup on this issue, I have now had an archive on a regular disk -- not a DMG, encrypted or otherwise -- since June 18th. That's about three weeks. Normally the problem would have come up by now, so I would conclude that QRecall has a problem with large archives on DMGs. Note that the problem could be an artifact of encryption, or it could just be an issue with DMGs. This particular archive is 830 GB and it contains about 55,000 files. As I mentioned before, I have other archives with many more files -- ballpark of 300,000 -- but they are much smaller, the bigger being 350 GB. To reproduce this on your end, try writing a script to create a set of 55,000-ish files, spread throughout multiple directories. Place the archive on an encrypted DMG. About 5000 files, emulating video, will vary between 500 MB and 1.2 GB. 50,000 files will emulate MP3 or AAC files, and will average between 5-10 MB each. Make the complete archive about 800 GB in size. Then start adding a few files every day to the archive, and after a couple weeks, you should see the problem. Maybe you can accelerate it by running the backup, adding files, then running the backup again, over and over. If you need some help writing a script to do this, I can probably knock something out in Ruby pretty fast to create the original files to be backed up, and then add a few files at random locations.
|
|
|
Hi James, thanks for the reply. I will move the music and video archive to a non-encrypted volume and see how that works for some time. It may take 2-4 weeks before you hear back, since the period over which the process breaks is usually about that long. Here's a theory: maybe iTunes changes something about one of the files being backed up during the backup process. There are two cases of potential importance, I think. One is that the file is changed after QRecall backs it up. This might cause QRecall to look at the file and die because it sees it's different. The second, probably more serious case, is where the actual file itself changes partway through the process of QRecall backing it up. In other word, the file contains n bytes. QRecall backs up x bytes, where x < n. Prior to QRecall finishing its backup, some program changes one of the bytes less than x, or worse, inserts multiple bytes or removes bytes (thereby making the file smaller or larger). QRecall completes the backup process and checks the file, and determines that the backup is inconsistent. Just some thoughts... steven
|
|
|
Hi James, I was not notified of your reply on the forum, but I did receive your private message, and I have successfully submitted my report to the QRecall Report Server using the "Send Report" mechanism. Also, in answer to your question, yes, all the archives are on a single encrypted disk image. steven
|
|
|
First of all, I love QRecall as a product. It's very sophisticated and allows me to backup to encrypted disk images, which is in fact what I do. (This may be part of the issue, bear this in mind. My archives reside on large encrypted disk images.) I have version 1.1 (1.1.0.42). If I should be trying a later version, let me know. The updater says I have the latest version. My problem comes because of one particular archive, my music and video archive. I have about 50,000 music files and 4500 video files. The archive I use for this tends to work correctly for a couple weeks, but then the index becomes corrupted. QRecall attempts to reindex the archive, but this inevitably fails, and it then has to rebuild the archive. I am not sure what the difference is, but both processes take about equally long -- and a very long time. I could separate this into two separate archives, one for video and one for music. But in terms of pure number of files, I probably add the same number of files to both sets per unit time, so I am concerned that, e.g. if the problem is related to the sheer number of files in the archive, maybe the video archive would be OK, but I'd encounter the same problem with the music archive after a couple weeks of updates. I don't really think it's about number of files, though, because I have a home directory archive that includes a source code directory and has nearly 300,000 files, and my documents directory, a separate archive, contains nearly that many again. So maybe it's about the size of the archives. Music and video is 781 GB; documents is 345 GB; home directory is 127.6 GB. Any suggestions on what I can do to fix, work around, or help the debugging process for this issue?
|
|
|
|
|