QRecallDownloadIdentity KeysForumsSupport
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Groups] Back to home page 
Archive size on APFS fills up disk in a strange way  XML
Forum Index » Problems and Bugs
Author Message
Johannes



Joined: 10-Dec-10 09:54
Messages: 66
Offline

(Not sure whether this is related with this topic: http://forums.qrecall.com/posts/list/913.page)

Since changing two Volumes from hfs+ to apfs I have two archives that frequently fill up my hard disk for apparently no reason.
After some investigation here are my findings:

  • Both archives report significant lower size in Finder than within QRecall (about 40 GB vs 120 GB).

  • Free Space of the disk indicates that Finder is wrong (like in the other thread)

  • Compacting the archive brings the archive back to the small size given in Finder

  • Trying to automate the solution with a compact action (condition: ignore if archive size is <60GB) failed. The action is not triggered (Finder size is 40 GB, Qrecall reports 72 GB today).I would assume that the condition is looking at the Finder size.

  • this did not happen when the Volmes were hfs+ formatted. Source volume and archive volume are both apfs.


  • Both archives are triggered several times daily, have regular merge actions and mostly have small layer sizes.

    I'll send a diagnostic report in a few minutes.

    Thanks for looking onto the issue.

    This message was edited 1 time. Last update was at 09-Jul-19 03:24

    James Bucanek



    Joined: 14-Feb-07 10:05
    Messages: 1434
    Offline

    Johannes,

    You are the third user that has reported something like this. And I'm beginning to suspect it's a bug in APFS. It also makes no sense to me that compacting the archive would make any difference.

    I would be interested in getting some allocation information about the archive files by running the 'ls -lskn' Terminal command on the archive (particularly at a point in time when the archive size and Finder size disagree), like this:

    Secondly, I'd be interested to know if the disk repair tool in Disk Utility produces any anomalous output when you repair that volume.

    Finally, I'd be curious to know if there are any snapshots of that volume. You can find out by issuing the command:

    - QRecall Development -
    [Email]
    Johannes



    Joined: 10-Dec-10 09:54
    Messages: 66
    Offline

    James,

    thanks for looking into this.

    Here is part 1 of the answers:

    Archiv 1:
    Finder size: 42,94 GB
    Qrecal size: 82,62 GB
    Output of ls -lskn:


    Archiv 2:
    Finder Size: 491,5 MB
    Qrecall Size: not available (opening the Archive presents an error as reported), probably 100 GB or more



    No snapshots on both Volumes.

    I'll try Disk repair later (have to save boot from an other account)

    Johannes
    James Bucanek



    Joined: 14-Feb-07 10:05
    Messages: 1434
    Offline

    What appears to be happening is that the repository.data file—the file with all the important data—is taking up more physical (allocated) space than it actually contains.

    In the case of Archiv 1, the repository.data file contained 40GB of data, but occupied 79GB of disk space. Stunningly, Archiv 2 contained only 0.4GB of data, but occupied 151GB of disk space.

    While adding sparse file support to QRecall 3.0, I've noticed that APFS can sometime over allocate space for a file and that is what seems to be happening here.

    My working theory is that APFS is not correctly handing pre-allocation requests. As the repository.data file grows during a capture, QRecall periodically makes pre-allocation requests at the end of the file so if the disk suddenly runs out of space, QRecall has enough "head room" to write its wrap-up metadata and session records.

    To test this theory, I've built a pre-release version of QRecall 2.1.16(1) that you can download and install. This hacked version doesn't perform any preallocation of the repository.data file. It won't fix the over-allocated space you have now, but if you compact the archive and perform new captures, those new captures won't cause it to over-allocate the file—assuming my theory is correct.

    Give this version and try and please keep me posted.

    - QRecall Development -
    [Email]
    Johannes



    Joined: 10-Dec-10 09:54
    Messages: 66
    Offline

    I compacted the Archive 1 (run surprisingly fast). Qrecall size is back to Finder size.
    I installed the special version of qrecall, opened the archive: Size in Qrecall is back to 80 something GB (strange)
    Again compacting (this time it took as long as usual). Qrecall Size is back to finder size.
    Now let's see what happens the next days ...

    Johannes











    Johannes



    Joined: 10-Dec-10 09:54
    Messages: 66
    Offline

    James,
    it seems your theory is correct. After 2 weeks both archives have kept their size. Everything works again as expected with the hacked version.
    Thanks for the quick support.
    Johannes
    James Bucanek



    Joined: 14-Feb-07 10:05
    Messages: 1434
    Offline

    Johannes,

    Thanks for the confirmation!

    Look for a new release of QRecall that works around this issue.

    - QRecall Development -
    [Email]
     
    Forum Index » Problems and Bugs
    Go to:   
    Powered by JForum 2.1.8 © JForum Team