James Bucanek wrote:I'm mystified. If you add up the sizes of those files it's clearly close to 20GB, and certainly not 150GB.

150GB was reported by "df", which includes the size of the other archive "uail.quanta" on the same volume.

James Bucanek wrote:Have you tried repairing the volume?

Good idea. Did it just now. And here is the terminal output.

Repairing file system.

Volume was successfully unmounted.

Performing fsck_apfs -y -x /dev/rdisk13s1

Checking the container superblock.

Checking the space manager.

Checking the space manager free queue trees.

Checking the object map.

Checking volume.

Checking the APFS volume superblock.

The volume zoo was formatted by newfs_apfs (748.31.8) and last modified by apfs_kext (945.250.134).

Checking the object map.

Checking the snapshot metadata tree.

Checking the snapshot metadata.

Checking snapshot 1 of 2.

error: directory valence check: directory (oid 0x3): nchildren (2) does not match drec count (0) 

warning: snapshot fsroot tree corruptions are not repaired; they'll go away once the snapshot is deleted

Checking snapshot 2 of 2.

error: directory valence check: directory (oid 0x3): nchildren (2) does not match drec count (0) 

Checking the extent ref tree.

Checking the fsroot tree.

error: directory valence check: directory (oid 0x3): nchildren (2) does not match drec count (0) 

Verifying allocated space.

Performing deferred repairs.

error: nchildren of inode object (id 3) does not match expected value

Restarting after deferred repairs...

Checking the space manager.

Checking the space manager free queue trees.

Checking the object map.

Checking volume.

Checking the APFS volume superblock.

The volume zoo was formatted by newfs_apfs (748.31.8) and last modified by apfs_kext (945.250.134).

Checking the object map.

Checking the snapshot metadata tree.

Checking the snapshot metadata.

Checking snapshot 1 of 2.

error: directory valence check: directory (oid 0x3): nchildren (2) does not match drec count (0) 

Checking snapshot 2 of 2.

error: directory valence check: directory (oid 0x3): nchildren (2) does not match drec count (0) 

Checking the extent ref tree.

Checking the fsroot tree.

Verifying allocated space.

The volume /dev/rdisk13s1 appears to be OK.



Operation successful.

The two snapshots were taken late last night and early this morning respectively. Per habit I take a system snapshot of the boot volume (with "sudo tmutil snapshot /" in terminal) before installing new software that I deem suspect or likely to be removed right away. There were indeed no snapshot on the volume when I started the thread last night. Because I didn't know tmutil would take a snapshot on all volumes (not just the boot/system volume), My follow-up message this morning didn't mention it either. My apologies.

There was, however, a third "nchildren (2) does not match drec count (0)" error not associated with snapshots. Diskutil repaired it. A second check after removing those two snapshots show no more errors.

Still, Finder (after restart) is reporting 90.91 GB available space, 4+GB less than 4 hours ago. "df" is reporting "153Gi" used (3Gi more than this morning), and "du" says docm.quanta is taking up 137G (2G more than this morning).

Yes, I'm mystified, too, especially when the other archive on the same volume seems to be unaffected. That one receives only daily update, though. So, I've just changed its capture schedule, now same with docm.quanta (3 min. after item change, with 21 min. hiatus after each capture). The source of the other archive is also busy for it includes my home (~) folder, so there will be a lot of actions. I'll report back later this evening.

The result of "ls -lhan /Volumes/zoo/docm.quanta":

total 279009064

drwxr-xr-x@ 22 501  80   704B May  3 09:49 .

drwxrwxr-x@  9 0    80   288B May  1 08:57 ..

-rw-r--r--   1 501  80    19K May  3 09:49 displayname.index

-rw-r--r--   1 501  80   7.2M May  3 09:49 filename.index

-rw-r--r--   1 501  80    98K May  3 09:49 fill.index

-rw-r--r--   1 501  80    96M May  3 09:49 hash.index

-rw-r--r--   1 501  80   112K May  3 01:29 hash_adjunct.index

-rw-r--r--   1 501  80   599K May  3 09:49 layer.index

-rw-r--r--   1 501  80    16M May  3 09:49 negative.index

-rw-r--r--   1 501  80   720B May  3 01:29 outline.index

-rw-r--r--   1 501  80   8.0M May  3 09:49 package.index

-rw-r--r--   1 501  80    16K May  3 09:49 package_adjunct.index

-rw-r--r--   1 501  80    17G May  3 09:49 repository.data

-rw-r--r--   1 501  80   8.3M May  3 09:49 repository_8k.checksum32

-rw-r--r--   1 501  80   1.0G May  3 09:49 repository_p8w8k16m2.0.anvin_reed_sol

-rw-r--r--   1 501  80   531K May  3 09:49 repository_p8w8k16m2.0_8k.checksum32

-rw-r--r--   1 501  80   1.0G May  3 09:49 repository_p8w8k16m2.1.anvin_reed_sol

-rw-r--r--   1 501  80   531K May  3 09:49 repository_p8w8k16m2.1_8k.checksum32

-rw-r--r--   1 501  80   122B May  3 09:49 sequence.index

-rw-r--r--   1 501  80   4.6K Apr 27 11:53 settings.plist

-rw-r--r--   1 501  80   866B May  3 09:49 status.plist

-rw-r--r--   1 501  80   5.4K Apr 18  2018 view.plist

Looks normal to me.

After one night's sleep, however, Finder now says the drive has only 95.21 GB available, 13.5G less than last night. "df" is reporting 150 Gi used, 13 Gi more than last night. And "du" is reporting 133G for "docm.quanta".

Don't know if it'll be useful, but I've sent in another report anyway, so that you can see what has been done overnight.

[edit] corrected a typo.

Hi,

Ran into something strange: one of my QRecall archive seems to be hiding disk space. The archive is using 20.18 GB according to Finder, but on the same Finder window you can see a 256GB drive with only two QRecall archives has only 108.71 GB available. (screenshot attached.)

There is no APFS snapshots ("sudo tmutil listlocalsnapshots" came back empty), and it's not indexed by Spotlight.

"df -h" shows:

/dev/disk11s1  238Gi  137Gi  101Gi    58%      87 9223372036854775720    0%   /Volumes/zoo

"du -h -d 1" shows:

16G	./uail.quanta

  0B	./.Trashes

288K	./.fseventsd

121G	./docm.quanta

  0B	./.TemporaryItems

137G	.

As you can see, the size of the other archive (uail.quanta) reported by du matches the number reported by Finder. The archive in question (docm.quanta), however, is taking up 121G, about 6 times as much as reported by Finder.

Verification shows no error. Compact would release the hidden space, on top of what "compact" would have saved normally.

I have been watching this for a while. Compacted it a few times, but the size would grow above 100G in a matter of days.

The archive backs up my ~/Documents folder, so it does capture often.

My system is macos 10.14.4, and my QRecall version is 2.1.14(6). I have another 5 QRecall archives on another drive. None of them have the issue.

A report has been sent.

Thanks!

[edit] the screenshot was of poor quality after shrinking. This one should be better.

Very happy to see v2.1 in beta. I'm you'll beta 31 on macos 10.13.4.

First bug to report: QRecall sometimes has troubles deleting items from an archive, taking a long time waiting for the archive to close. (screenshot 1)

The dialog would say "complete" if I wait long enough, but it won't go away, and the "Stop" button would be grayed out and unclickable. (screenshot 2)

When that happens, QR has to be force-quit.

I've run into the problem numerous times with two different archives, so it doesn't seem to be archive-specific. I've deleted items from both archives successfully as well, so it seems random to me so far.

A report has been sent.

edit: the forum is acting funny and the 2nd attachment is shown on top.

Hi James,

Congratulations on the v2 release. QRecall has been one of my favorite tools since I discovered it a little more than a year ago, and now it's the first program I install when setting up a new system. Having followed through the v2 beta process, moreover, I must say I admire your work ethic, especially the methodical way you tackle bugs.

No software is perfect and I'm sure to bring you more demands later; I believe you have your plan as well. But QR2 is truly a gem and the best backup tool for Mac (and I've tried many).

Thank you and, again, congratulations!

Doh! False alarm. Finally got it. Not sure since when, but QR has created another new volume for the partition. I always combine them when I see that, but was unaware of this one. Turned out I was looking at the wrong volume.

My mistake; my sincere apologies.

1. Forgot to say: verification turned up no error.

2. Forgot to ask: the archive is set to keep deleted items for 0 days. Why are deleted items still there after being compacted?

UPDATE: the following is kept for the record, but it's a false alarm (see the 3rd post below).


It's been a while since I last checked in here because QR2 betas have been doing well on my system. Or so I thought, until this morning when I checked out my system archive (the one backing up my system partition).

I can't believe what I saw: the contents of the /Applications folder is different--very different--from the real ones. Applications that have been deleted are still in there (and not marked as deleted), while some that should be there are missing.

There are 4 different types of discrepancies (screenshots below show the first two):

1. Deleted Applications still shown as active

"1Password 6", "3DGallery", "A Better File Rename 10", "Adobe Digital Editions", "Adobe Digital Editions 4.5" are all gone from my system, but QR "thinks" they are current.

2. Active applications shown as deleted

1Password is marked as deleted in QR, but in fact it's active.

(note: 1Password and "1Password 6" are the same thing, except the former is from the App Store. I bought from the App Store--a decision I regret, btw--and downloaded 1Password 6 from their website while it's in beta. Changed back to the App Store version when it's officially released.)

3. Active applications not backed up

4. Applications contents not updated

I let QR to recall the whole Applications folder to a different location and did a full binary comparison. Many files are different. The most current backup was captured earlier this morning (at 3am), and that's the one I recalled. I certainly haven't installed or updated anything since then, and some applications haven't been used for days if not weeks. MAS auto update is turned off on my system, and those applications are not running in the background.

My system is El Capitan 10.11.3.

James Bucanek wrote:This should be fixed in 2.0.0b26 (just released).

Indeed. Many thanks!

James Bucanek wrote:What exactly do you mean by "it didn't work"? Looking at the logs, I don't see anything that would indicate a problem opening the archive. Was the window just blank? Was it taking forever to display the layers? ... ?

Sorry for not making it clear in the first place: by "it didn't work" I meant nothing happened after I double-clicked the archive in the "Open Archive..." dialog (or clicked the "Open" button after selecting the archive). The dialog would close and that's it. Nothing else happened.

Tried it a few times just now, and made sure to wait more than 15 min. for it. Still nothing. Had to double-click in the Finder to open in able to conduct the next test. If a stale lock is the cause, wouldn't it also prevent the archive from opening right away from Finder?

Be patient. Once an archive acquires an orphaned access lock, it take a while for QRecall to determine that the lock is stale and reset it. It does this automatically, but it takes about 10 minutes for QRecall assure itself that there are no other processes accessing the archive at a the same time.

OK, I tried to delete an item in the archive just now, and have been patiently waiting for it. More than 15 min. time has passed and QR is still "waiting for archive".

A different message did came up (replacing the "waiting for archive" message) for a split second, too short for me to read it as "waiting for archive" reappeared right away.

I suspect all is well now. You waited the magic 10 minutes for QRecall to determine that it was safe to access the archive. Once it had, the access lock for the archive was reset and actions will now proceed normally.

Unfortunately, no. I conducted several tests after the successful compact action last night before reporting. Did the same this morning.

While "waiting for archive" when trying to delete an item, an event-driven capture to the same archive was triggered. That action had to "wait for archive" as well, so I stopped the new (capture) action. That gave me an idea: after canceling the delete job and quitting QR to close everything, I reopened QR and made a manual capture from the Actions window. The capture waited about 5 min. before going into action. The is probably due to the stale lock you mentioned.

The next capture would go swiftly as usual, so the stale lock should have been cleared. Still can't open the archive from the "Open Archive..." dialog.

Forgot to add (though you might have guessed): no other archive has the issue. This archive is not large (some others are much larger), currently taking up 7+GB, but it has the greatest number of layers (135 and counting), if that matters.

QR2 beta 25 on El Capitan 10.11.1.

Today when trying to open an archive from "File - Open Archive ..." dialog, it didn't work (it's been a while since I last opened the archive, so it's no longer on the recent archives list). Double clicking on the archive in Finder did work, but then it can't be closed, either by clicking on the window control button or with the "File - Close" command (or cmd-W, for that matter). Had to quit QR to close it. I've tried it several times with the same result. I even tried it once on another desktop (running the same beta version, but on Yosemite); same result.

The problem is not limited to opening and closing. I can't delete any item on it, nor can I remove any layer, merge layers, compact the archive or combine volumes. (The archive has two volumes of the same source directory after I installed El Capitan, so I want to combine them.) The action would appear to be starting, but would be "waiting for archive" to no end.

After clicking "Stop" to cancel the action, there's an entry in the log that says "Problem encountered while closing archive", so it appeared to be related to the closing problem after all.

Verifications (numerous times) found no error. Regular captures seem to be fine as well. I also successfully compacted it from the Actions window without opening the archive, but the action waited more than 10 min. before really doing anything.

A diag report has been filed.

I ran into two small issues when trying to restore the whole system partition with QR 2.0 beta for the first time yesterday. The restoration was done from a standby system (a bare minimum El Capitan system with QR 2.0 b23 and some other essential utilities installed) as I intended to restore the whole system partition.

The restoration was mostly successful, but QR said it failed at the end. The log registered two errors:

Action	2015-11-27 12:53:00	       	  Recall issues

Action	2015-11-27 12:56:57	       	    volume carlosys

Action	2015-11-27 12:56:57	       	      Applications

Action	2015-11-27 12:53:00	       	        folder CrashPlan.app

Action	2015-11-27 12:53:00	Caution	          Stopped processing folder

Action	2015-11-27 12:53:00	       	            unable to set attributes

Action	2015-11-27 12:53:00	       	              Type: directory

Action	2015-11-27 12:53:00	       	              Path: /Volumes/carlosys/Applications/CrashPlan.app

Action	2015-11-27 12:53:00	       	              Name: CrashPlan.app

Action	2015-11-27 12:53:00	       	              ErrDescription: Operation not permitted

Action	2015-11-27 12:53:00	       	              attributeSet: 0x00020f00

Action	2015-11-27 12:53:00	       	              POSIXErr: 1

Action	2015-11-27 12:56:57	       	      Users/wang

Action	2015-11-27 12:56:40	       	        folder Documents

Action	2015-11-27 12:56:40	Caution	          unable to remove access control list

Action	2015-11-27 12:56:40	Minutia	            POSIXErr: 22

Action	2015-11-27 12:56:40	       	            ErrDescription: Invalid argument

I tried to restore again for the user profile folder (Users/wang) and this time QR made no complaint. Trying to do the same with the CrashPlan.app however ran into the same issue. I ended up reinstalling CrashPlan after booting into the restored system.

The system has been running smoothly after restoration so there should be no problem. But I guess I should report the incident back to you. A diag report was filed right after the failure yesterday.

Thank you. The old, problematic system archive of mine is now back in service.

James Bucanek wrote:I do have one more question: did you unmount the volume (mount point) at ~/Documents before you made the capture, or before you performed the verify?

Sorry for not making it clear earlier. By "trying again after unmounting the ~/Documents partition" I meant the whole thing: creating a new test archive, capturing my user profile directory tree, and then verifying the archive.