QRecallDownloadIdentity KeysForumsSupport
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Groups] Back to home page 
Messages posted by: Ming-Li Wang
Forum Index » Profile for Ming-Li Wang » Messages posted by Ming-Li Wang
Author Message
10 hours have passed since the drive hosting those archives have been reformatted into HFS+, and there is no sign of either archive taking up any ghost space (checked in Finder and with df & du, as usual). It's now pretty clear it's APFS-related (& most likely an APFS bug). Thanks to Adrian for bringing the possibility to my attention, though I have no clue if/how it's related to the issues mentioned in the discussion thread you provided.

I had decided to leave the volume as is, but an itch made me give APFS one final chance. This time, I erased the whole drive (yesterday I merely reformatted the APFS volume without touching the container/partition). I'll report back if I find something interesting.
James Bucanek wrote:So if this is the discrepancy, it might be something you can just ignore.

It's not. I thought I have made that clear. I was only arguing that it's not useless to take snapshots of a volume dedicated to hosting QRecall archives.

I'm going out in a min., will report my findings after I'm back.
James Bucanek wrote:Because of the issues I've encountered with APFS volumes getting corrupted, and since you mentioned that you have the space available to move the archives to a different volume, I'd suggest copying the archive to another volume, repartition and reformat the APFS volume, then move the archives back.

It's what I did earlier this evening, and it didn't help as said in the previous post (before I saw your latest post).

James Bucanek wrote:Also, you mentioned that "I take a system snapshot of the boot volume", but we're talking about the volume the archives are on, right? There shouldn't be any snapshots of 'zoo'. (It doesn't make any sense to make a snapshot of an archive volume, since archives are literally a collection of snapshots/layers.) If there are, that could be the problem—at least the root of the free space problem.

Yes, I meant only to take a snapshot of the boot volume (with "sudo tmutil snapshot /"). I didn't know tmutil actually took a snapshot for each active volume. (Well, I didn't check them all, but I did check a couple and they all had a snapshot of the same timestamp.)

That being said, I don't think it's useless to take snapshot of a volume dedicated for backups, as system snapshot and cloud backup are the only two means I know that can withstand a ransomware attack.
OK, what I did a few hours earlier doesn't seem to help as docm.quanta is still putting up weight. (I changed uial.quanta's schedule back to one capture per day, so it's fine.)

Given your conversation above, I'll try something new tonight. I moved the two archives away, again, reformatted the drive into HFS+, and moved the archives back. We should find out if APFS is the culprit tomorrow morning. (It's 1am local time, and I'm going to bed.)
Less than 5 hours into my experiment, both archives have put up substantial weight:

Apparently the issue is not archive specific.

Other than compacting both archives, I decided to try something different this time. I moved both of them to another drive (drive B), reformatted the original drive (drive A), and then move them back.

As soon as the first "move" action completed, drive A got all its missing space back according to Finder, concurred by df, and du. On drive B, "du" reported normal size for both:

As expected, they are still normal after being moved back to drive A.

Another report has been sent.
James Bucanek wrote:I'm mystified. If you add up the sizes of those files it's clearly close to 20GB, and certainly not 150GB.

150GB was reported by "df", which includes the size of the other archive "uail.quanta" on the same volume.

James Bucanek wrote:Have you tried repairing the volume?

Good idea. Did it just now. And here is the terminal output.

The two snapshots were taken late last night and early this morning respectively. Per habit I take a system snapshot of the boot volume (with "sudo tmutil snapshot /" in terminal) before installing new software that I deem suspect or likely to be removed right away. There were indeed no snapshot on the volume when I started the thread last night. Because I didn't know tmutil would take a snapshot on all volumes (not just the boot/system volume), My follow-up message this morning didn't mention it either. My apologies.

There was, however, a third "nchildren (2) does not match drec count (0)" error not associated with snapshots. Diskutil repaired it. A second check after removing those two snapshots show no more errors.

Still, Finder (after restart) is reporting 90.91 GB available space, 4+GB less than 4 hours ago. "df" is reporting "153Gi" used (3Gi more than this morning), and "du" says docm.quanta is taking up 137G (2G more than this morning).

Yes, I'm mystified, too, especially when the other archive on the same volume seems to be unaffected. That one receives only daily update, though. So, I've just changed its capture schedule, now same with docm.quanta (3 min. after item change, with 21 min. hiatus after each capture). The source of the other archive is also busy for it includes my home (~) folder, so there will be a lot of actions. I'll report back later this evening.
The result of "ls -lhan /Volumes/zoo/docm.quanta":

Looks normal to me.

After one night's sleep, however, Finder now says the drive has only 95.21 GB available, 13.5G less than last night. "df" is reporting 150 Gi used, 13 Gi more than last night. And "du" is reporting 133G for "docm.quanta".

Don't know if it'll be useful, but I've sent in another report anyway, so that you can see what has been done overnight.

[edit] corrected a typo.

Ran into something strange: one of my QRecall archive seems to be hiding disk space. The archive is using 20.18 GB according to Finder, but on the same Finder window you can see a 256GB drive with only two QRecall archives has only 108.71 GB available. (screenshot attached.)

There is no APFS snapshots ("sudo tmutil listlocalsnapshots" came back empty), and it's not indexed by Spotlight.

"df -h" shows:

"du -h -d 1" shows:

As you can see, the size of the other archive (uail.quanta) reported by du matches the number reported by Finder. The archive in question (docm.quanta), however, is taking up 121G, about 6 times as much as reported by Finder.

Verification shows no error. Compact would release the hidden space, on top of what "compact" would have saved normally.

I have been watching this for a while. Compacted it a few times, but the size would grow above 100G in a matter of days.

The archive backs up my ~/Documents folder, so it does capture often.

My system is macos 10.14.4, and my QRecall version is 2.1.14(6). I have another 5 QRecall archives on another drive. None of them have the issue.

A report has been sent.


[edit] the screenshot was of poor quality after shrinking. This one should be better.
Very happy to see v2.1 in beta. I'm you'll beta 31 on macos 10.13.4.

First bug to report: QRecall sometimes has troubles deleting items from an archive, taking a long time waiting for the archive to close. (screenshot 1)

The dialog would say "complete" if I wait long enough, but it won't go away, and the "Stop" button would be grayed out and unclickable. (screenshot 2)

When that happens, QR has to be force-quit.

I've run into the problem numerous times with two different archives, so it doesn't seem to be archive-specific. I've deleted items from both archives successfully as well, so it seems random to me so far.

A report has been sent.

edit: the forum is acting funny and the 2nd attachment is shown on top.
Hi James,

Congratulations on the v2 release. QRecall has been one of my favorite tools since I discovered it a little more than a year ago, and now it's the first program I install when setting up a new system. Having followed through the v2 beta process, moreover, I must say I admire your work ethic, especially the methodical way you tackle bugs.

No software is perfect and I'm sure to bring you more demands later; I believe you have your plan as well. But QR2 is truly a gem and the best backup tool for Mac (and I've tried many).

Thank you and, again, congratulations!
Doh! False alarm. Finally got it. Not sure since when, but QR has created another new volume for the partition. I always combine them when I see that, but was unaware of this one. Turned out I was looking at the wrong volume.

My mistake; my sincere apologies.
1. Forgot to say: verification turned up no error.

2. Forgot to ask: the archive is set to keep deleted items for 0 days. Why are deleted items still there after being compacted?
UPDATE: the following is kept for the record, but it's a false alarm (see the 3rd post below).

It's been a while since I last checked in here because QR2 betas have been doing well on my system. Or so I thought, until this morning when I checked out my system archive (the one backing up my system partition).

I can't believe what I saw: the contents of the /Applications folder is different--very different--from the real ones. Applications that have been deleted are still in there (and not marked as deleted), while some that should be there are missing.

There are 4 different types of discrepancies (screenshots below show the first two):

1. Deleted Applications still shown as active

"1Password 6", "3DGallery", "A Better File Rename 10", "Adobe Digital Editions", "Adobe Digital Editions 4.5" are all gone from my system, but QR "thinks" they are current.

2. Active applications shown as deleted

1Password is marked as deleted in QR, but in fact it's active.

(note: 1Password and "1Password 6" are the same thing, except the former is from the App Store. I bought from the App Store--a decision I regret, btw--and downloaded 1Password 6 from their website while it's in beta. Changed back to the App Store version when it's officially released.)

3. Active applications not backed up

4. Applications contents not updated

I let QR to recall the whole Applications folder to a different location and did a full binary comparison. Many files are different. The most current backup was captured earlier this morning (at 3am), and that's the one I recalled. I certainly haven't installed or updated anything since then, and some applications haven't been used for days if not weeks. MAS auto update is turned off on my system, and those applications are not running in the background.

My system is El Capitan 10.11.3.
James Bucanek wrote:This should be fixed in 2.0.0b26 (just released).

Indeed. Many thanks!
James Bucanek wrote:What exactly do you mean by "it didn't work"? Looking at the logs, I don't see anything that would indicate a problem opening the archive. Was the window just blank? Was it taking forever to display the layers? ... ?

Sorry for not making it clear in the first place: by "it didn't work" I meant nothing happened after I double-clicked the archive in the "Open Archive..." dialog (or clicked the "Open" button after selecting the archive). The dialog would close and that's it. Nothing else happened.

Tried it a few times just now, and made sure to wait more than 15 min. for it. Still nothing. Had to double-click in the Finder to open in able to conduct the next test. If a stale lock is the cause, wouldn't it also prevent the archive from opening right away from Finder?

Be patient. Once an archive acquires an orphaned access lock, it take a while for QRecall to determine that the lock is stale and reset it. It does this automatically, but it takes about 10 minutes for QRecall assure itself that there are no other processes accessing the archive at a the same time.

OK, I tried to delete an item in the archive just now, and have been patiently waiting for it. More than 15 min. time has passed and QR is still "waiting for archive".

A different message did came up (replacing the "waiting for archive" message) for a split second, too short for me to read it as "waiting for archive" reappeared right away.

I suspect all is well now. You waited the magic 10 minutes for QRecall to determine that it was safe to access the archive. Once it had, the access lock for the archive was reset and actions will now proceed normally.

Unfortunately, no. I conducted several tests after the successful compact action last night before reporting. Did the same this morning.

While "waiting for archive" when trying to delete an item, an event-driven capture to the same archive was triggered. That action had to "wait for archive" as well, so I stopped the new (capture) action. That gave me an idea: after canceling the delete job and quitting QR to close everything, I reopened QR and made a manual capture from the Actions window. The capture waited about 5 min. before going into action. The is probably due to the stale lock you mentioned.

The next capture would go swiftly as usual, so the stale lock should have been cleared. Still can't open the archive from the "Open Archive..." dialog.

Forum Index » Profile for Ming-Li Wang » Messages posted by Ming-Li Wang
Go to:   
Powered by JForum 2.1.8 © JForum Team