QRecall Community Forum
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Top Downloads] Top Downloads   [Groups] Back to home page 
[Register] Register /  [Login] Login 

Crash during error correction inflation RSS feed
Forum Index » Beta Version
Author Message
Ralph Strauch


Joined: Oct 24, 2007
Messages: 194
Offline
I have two drives containing essentially the same archives, one of which is normally stored offsite. I followed your direction to have both of them mounted when I first ran v2.0. I then started a backup into one of the drives, and looking at things like the archive settings on the other, and decided I could probably turn on Error Correction on the second archive while I was still backing up on the first. Initially, that seemed to work, but at about 56gb in to a 1tb archive, qrecall crashed and the error correction froze. The backup on the first drive is still running, so I'm assuming that it will complete satisfactorily. (Activity Monitor shows Qrecall as "not responding" and shows three instances of Qrecall Helper running, two active and one inactive. I don't want to force quit quit while the backup is running, but once it finishes I'll restart qrecall and send you a report. I'm running OSX 10.11.1 on a 2014 rMBP.

Question: Once I get this computer backed up to both archives I have a second computer to back up to both as well. Should that computer, as well, have both drives mounted when I run v2 for the first time?
James Bucanek


Joined: Feb 14, 2007
Messages: 1567
Offline
Ralph Strauch wrote:(Activity Monitor shows Qrecall as "not responding" and shows three instances of Qrecall Helper running, two active and one inactive. I don't want to force quit quit while the backup is running, but once it finishes I'll restart qrecall and send you a report.

It sounds like both actions might still be running. A capture runs as two instances of QRecallHelper (one to perform the capture and a second to gather user-centric metadata; the latter is usually not very busy), and the error correction task runs a single instance. So my advice is to let them run. Eventually the QRecallHelper processes will run their course and you can check the log for success, failure, or incompleteness.

Why the QRecall app itself it hung is a mystery. If you have the Activity monitor open and QRecall is still "not responding" take a sample of it and email it to me, if you have the time.

Question: Once I get this computer backed up to both archives I have a second computer to back up to both as well. Should that computer, as well, have both drives mounted when I run v2 for the first time?

It's easiest if your archive are all reachable when QRecall 2.0 is run for the first time, but it is certainly not a requirement. You can easily make these updates yourself at anytime. Here's what the "upgrade" process consists of, in a nutshell:

  • Replace the legacy alias structure in the action document with a modern URL bookmark. You can do this by opening each action, reselecting the archive, and saving it again.

  • Move any excluded items from the archive's capture action to the archive's settings. To do this yourself, simply open the archive and edit its settings to set the you items want excluded.

  • Move any ignored items from the archive's capture action and turn them into capture preferences for that item. To do that yourself, choose the items you want ignored in the Finder and use the new QRecall Capture Preferences service.


  • Finally, aliases can be finicky. Even if you let QRecall perform the 2.0 upgrade, you might have better success with your rotating set of archives if you manually open their actions, reselect the archive, and save them again. That way, you'll know QRecall is storing the correct bookmark for each archive.

    - QRecall Development -
    [Email]
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    Here's the rest of the story. The backup continued to run and I watched its progress in Qrecall Monitor. When it got about 1/8" from the end of the status bar it stopped progressing. I checked Activity Monitor and it showed Qrecall Monitor also "not responding. so I force-quit both Qrecall and Monitor. I tried to restart Qrecall and couldn't, so I shut down and restarted the computer. As I was doing this I found most of the other apps I was running had also frozen and I ended up doing a hard power down reset. I don't know whether Qrecall was the cause of all of this, or it was just coincidental.

    I'm back up now and it looks like my backup was completed, according to the log. I just sent a Report. At some point I'll try to verify the backup and see what I get, but I may go ahead first with a backup to my second archive, since the backup operation itself seems OK. I won't do anything more to install Error Correction until I hear from you.
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    I just tried to backup to my second archive, the one where I had the problem installing Error Correction. Qrecall now tells me that that archive needs reindexing, so I've just started a repair. Qrecall Monitor is showing Idle, while Qrecall is showing the repair in progress. I don't remember if that's standard, or not.
    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1567
    Offline
    Ralph Strauch wrote:As I was doing this I found most of the other apps I was running had also frozen and I ended up doing a hard power down reset. I don't know whether Qrecall was the cause of all of this, or it was just coincidental.

    I'm not aware of anyway that QRecall could cause your other apps to hang, so I'm going to go with "coincidence."

    - QRecall Development -
    [Email]
    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1567
    Offline
    Ralph Strauch wrote:I just tried to backup to my second archive, the one where I had the problem installing Error Correction. Qrecall now tells me that that archive needs reindexing, so I've just started a repair.

    According to the log you sent me, the capture action completed but it looks like the error correction change was still running when your restarted your computer. A reindex should fix it.

    Qrecall Monitor is showing Idle, while Qrecall is showing the repair in progress. I don't remember if that's standard, or not.

    That's normal. Actions that modify an archive run in the archive browser window, if that's where you started it.

    - QRecall Development -
    [Email]
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    It's been a busy evening. I started to write this response before your last reply, which came in the middle of the chaos described below.

    The repair I started on the drive with the Error Correction problem has been running for almost six hours and I just force-quit it because it's generated a log of more than 45GB so far and if I don't turn it off before I got to bed it will fill my drive before I get up. It seemed to go through the first part of the repair cycle normally, then when it switched to "reindexing . . ." it started generating lines in the log that shows up in the app like this

    Action 2015-10-30 21:40:36 Warning ignoring problem reading checksum; assume data is valid
    Action 2015-10-30 21:40:36 problem reading checksums
    Action 2015-10-30 21:40:36 Position: 483863340
    Action 2015-10-30 21:40:36 Length: 4

    thousands of them, one after another. The basic interval between one Position and the next was 4, the position number itself changed from one block to another from time to time. When TechTool Pro gave em a low disk space warning I checked and saw my disk space disappearing at a notable rate. When I shut it down the Qrecall Log in ~/Library was over 45gb. Qrecall didn't respond to the stop button, so I had to force quit it.

    The log in the app now shows only a segment of the repetitive lines; the earlier stuff before that didn't survive the shut down. I started to send a report, but stopped it when it was still "compressing" after half an hour. I was in the middle of watching something on TV, running back and forth during commercials. When I came back at the end of the show, Activity Monitor showed qrecall still taking a lot of cpu, then shortly thereafter I got a system message that I was out of system memory because of Qrecall and needed to force quit applications, most of which were showing "paused" (including Safari, where I had most of this post written. Luckily I was able to resume it.)


    So my current situation is that I'm sitting here with my backup drive unplugged and Qrecall not running. If I mount the drive it makes clicking noises even though nothing on the computer seems to be activating it. If I start Qrecall (with the drive mounted or not) and do nothing else with it, Activity Monitor shows it as "not responding" and gobbling up increasing amounts of memory (over 20gb not doing anything.

    This computer now shows the 45gb Qrecall log generated while the archive repair was reindexing, along with a couple of smaller logs generated after that. Let me know if you would like any of that material. or other information I can provide. I'm about to head off to bed now, and will pick this up in the morning. I'm thinking now that my next step should be to reinstall qrecall and see if that lets it start without immediately gobbling up memory. Is that a good starting point, or not?

    If I can get qrecall running properly again, then I can worry about whether this archive can be save, or will just have to be junked. I'm very glad right now that I had a pair of backup drives, and ran a backup using v2.0 on the other drive before these problems occurred. It's mounted on my other computer right now doing a Verify, which I hope will come out OK.
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    I just started qrecall to see what would happen. Watching it on Activity Monitor it just began to gobble up memory, even though I wasn't asking it to do anything. Sample from Activity Monitor attached. By the time I force-quit it, it was using 32gb of memory, I only have 16.
     Filename Sample of QRecall 20151031.txt [Disk] Download
     Description Sample from qrecall this morning
     Filesize 45 Kbytes
     Downloaded:  668 time(s)

    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1567
    Offline
    Ralph,

    Don't worry. You're the unfortunate victim of the unfinished action that was adding error correction codes to the archive. Nothing you've done has made any changes, or endangered, the data in your primary data file.

    Here's what happened. The action to add error correction codes to your archive crashed (or was terminated during the reboot). The correction code files are now incomplete. When you tried to reindex the archive, QRecall—in the interest of full disclosure—started logging every missing checksum and error correction block it found, or didn't find in this case.

    That caused your log to explode. When you opened the QRecall app, you probably had the log window open. QRecall started reading the gigabytes worth of new log messages into memory. This began to eat up all of you memory and drag your system into the mud.

    Here's how to get out of this mess:

    First, get rid of the bloated log file. Open your ~/Library/Logs/QRecall folder and trash anything that looks too big. That's probably just going to be your primary QRecall.log file, unless you rotate log files daily.

    Select your archive in the Finder. Control/Right-click on the archive and choose Show Package Contents. Inside the archive you're going to find a repository.data file. You'll also see a bunch of "companion" files with funny names like this:

    repository_8k.checksum32
    repository_p8w8k16m2.0.anvin_reed_sol
    repository_p8w8k16m2.0_8k.checksum32
    repository_p8w8k16m2.1.anvin_reed_sol
    repository_p8w8k16m2.1_8k.checksum32


    Trash all of these "companion" files. (Do not trash your repositoy.data file. That's where all of your data lives!)

    Launch QRecall. Choose File > Repair Archive…, choose your archive, select Reindex, and Bob's your uncle.

    - QRecall Development -
    [Email]
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    I threw away the files you suggested and repaired the archive, and everything seems fine. I'm doing another backup now. The repair log listed about 3 dozen instances of "invalid data," mostly just a few hundred bytes but one chunk of half a megabyte. Should I be concerned about these at all.

    Qrecall continues to be a great app, and one of it's finest features is the quality and responsiveness of the support that comes with it. Thanks again, James, for all you provide.

    Ralph
    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1567
    Offline
    Ralph Strauch wrote:I threw away the files you suggested and repaired the archive, and everything seems fine. I'm doing another backup now. The repair log listed about 3 dozen instances of "invalid data," mostly just a few hundred bytes but one chunk of half a megabyte. Should I be concerned about these at all.

    What you really want to look for are indications of incomplete folders or files that were a result of those stretches of "invalid data". These will also be in the log. If you don't see any messages like that, then the stretches of invalid data were probably detritus from an incomplete merge or deleted items (as a result of an earlier merge) that hadn't yet been erased.

    Qrecall continues to be a great app, and one of it's finest features is the quality and responsiveness of the support that comes with it. Thanks again, James, for all you provide.

    You're very welcome.

    - QRecall Development -
    [Email]
    Ralph Strauch


    Joined: Oct 24, 2007
    Messages: 194
    Offline
    Is it safe to make another try at adding error correction now, or should I wait for the next release?
    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1567
    Offline
    Ralph Strauch wrote:Is it safe to make another try at adding error correction now, or should I wait for the next release?

    As far as I can tell, your issues with adding error correction was in the QRecall app (the monitor), not the action (the doer). It appears that the action continued to run (until the computer was restarted), as I can find no crash logs or other evidence that it died abnormally.

    I'd suggest you give it another run. Just remember that it has to read the entirety of the archive data, so it will take about as long as a verify.

    - QRecall Development -
    [Email]
     
    Forum Index » Beta Version
    Go to:   
    Mobile view
    Powered by JForum 2.8.2 © 2022 JForum Team • Maintained by Andowson Chang and Ulf Dittmer