Any issues backing up data from Xsan volumes?... or as archive destinations?

Seems to be working fine archiving a small Project folder. FCP, PSD AE etc...

I then pointed it to all our projects, about 700GB, and eventually got a out of disk space error... it had 20TB available.

I'm using the 12day demo.

Any way to include or exclude file types by extension?

Thanks
John

John Heagy wrote:Any issues backing up data from Xsan volumes?... or as archive destinations?

QRecall is specifically designed to capture files on local, HFS+ formatted volumes. It works, with limitations, capturing other filesystems (MS-DOS, UFS, etc.). It can capture files on remote/networked volumes, but will be limited to the security and access restrictions imposed by the server.

The archive itself is simple and only uses flat POSIX-compatible files. You should be able to create an archive on any filesystem supported by Mac OS X that supports large files (a SAN device, a USB thumbdrive, ...). I will note that QRecall does not play well with WebDAV volumes. Search the forum archives for more info.

Seems to be working fine archiving a small Project folder. FCP, PSD AE etc...

I then pointed it to all our projects, about 700GB, and eventually got a out of disk space error... it had 20TB available.

A common problem with many external and networked volumes is that they may be using a volume format or file server that does not support files greater than 4GB. What you'll see is QRecall capture about 3.9 GB of data and then stop with an error that it ran out of disk space. (I plan to include a test for this specific situation in a future version, but for now this is the behavior.)

You need to determine if the volume format and file sharing protocols you're using supports files larger than 4GB. If not, then looking into reformatting the volume (preferably HFS Extended) or upgrading your file server software/firmware. For example, the original Apple Time Capsule requires a firmware upgrade to work with 4GB files.

I'm using the 12day demo.

A trial key is fully functional.

Any way to include or exclude file types by extension?

Not in the current version. This is on the to-do list for an upcoming release.

A common problem with many external and networked volumes is that they may be using a volume format or file server that does not support files greater than 4GB. What you'll see is QRecall capture about 3.9 GB of data and then stop with an error that it ran out of disk space. (I plan to include a test for this specific situation in a future version, but for now this is the behavior.)

You need to determine if the volume format and file sharing protocols you're using supports files larger than 4GB. If not, then looking into reformatting the volume (preferably HFS Extended) or upgrading your file server software/firmware. For example, the original Apple Time Capsule requires a firmware upgrade to work with 4GB files.

Hmm... our Xsan is certainly not limited to any file size and is fibre attached, so it's a local HD as far as any apps are concerned. It does use a clustered file system as any San would.

I tried another smaller archive and it complained of lack of disk space when attempting to archive 300GB to 5TB of free space after writing only 5.8GB. I suppose it doesn't like writing to an XSan.

Are you aware of any customer using QRecall to archive files to an Xsan volume? I my test case it's from one Xsan volume to another.

Thanks
John

John Heagy wrote:Hmm... our Xsan is certainly not limited to any file size and is fibre attached, so it's a local HD as far as any apps are concerned. It does use a clustered file system as any San would.

I tried another smaller archive and it complained of lack of disk space when attempting to archive 300GB to 5TB of free space after writing only 5.8GB. I suppose it doesn't like writing to an XSan.

That's possible. If you're not running into the 4GB file size limit, then something else is amiss. Send me a diagnostic report (Help > Send Report...) and I'll look into it further.

Are you aware of any customer using QRecall to archive files to an Xsan volume? I my test case it's from one Xsan volume to another.

Not that I'm aware of, although I know many people are using various NAS drives without any problems. And I capture to a 3TB RAID system daily.

John,

One more question: Are you running QRecall on the Xsan metadata controller or from one of the clients? If so, what's the version of Mac OS X running on the controller (the diagnostic report will tell me what OS is running on the client)?

QRecall is running on an Xsan client. The MDC is at 10.5.7

John,

Congratulations, you've discovered a new problem! That's not what most customers want to hear, but there it is.

During a capture, QRecall uses a feature of the operating system to pre-allocate file space on the volume. This protects it from "painting itself into a corner" by ensuring that there's always enough free space on the volume to close the archive and write all the necessary index files.

For some reason, QRecall's attempt to pre-allocate space for one of your index files is failing. This is from your log file:

2009-08-27 18:39:14.027 -0400 #debug# preallocation failed

2009-08-27 18:39:14.027 -0400 #debug# API: FSAllocateFork

2009-08-27 18:39:14.028 -0400 #debug# OSErr: -48

2009-08-27 18:39:14.028 -0400 Details Length: 36864

2009-08-27 18:39:14.028 -0400 Details Pos: 1081392

2009-08-27 18:39:14.028 -0400 Details File: /Volumes/Xsan1a/Edit/EVS/QRecall/ProjectsA.quanta/package_scribble.index

2009-08-27 18:39:14.028 -0400 #debug# RepositoryPackage received kPreallocationFailedNotification

2009-08-27 18:39:14.442 -0400 #debug# caught exception

2009-08-27 18:39:14.442 -0400 Details disk full

The error that QRecall is getting back from the operating system when it tries to preallocate a mere 36K of additional disk space is -48 (duplicate file error), which frankly doesn't make any sense at all. QRecall assumes that you've run out of disk space simply because the error was returned from the pre-allocate request, which is just about the only reason that call should ever fail.

I suspect that this is a bug in Apple's server OS or XSAN implementation. The FSAllocateFork has had a checkered history of bugs (I've filed at least three), some of which have were fixed in OS X 10.5 (client). Other's are still outstanding.

QRecall already has a workaround for OS X 10.4, where the FSAllocateFork function doesn't work correctly at all. I could send you a special version of QRecall that uses that workaround all the time, or a special version that doesn't attempt to pre-allocate at all. With 5TB of free space, you clearly don't need this mechanism; it's really to protect archives on really small volumes, like USB thumb drives.

If you're interested in testing a special version, let me know an I'll code it up and send it to you.

If you're interested in testing a special version, let me know an I'll code it up and send it to you.

Sure... that would be great!

Thanks a bunch
John

John,

I was wondering what success you'd had with the Xsan and 1.1.3a5, and if you were successful in opening an archive in a read-only folder?

Hi James,

I suppose you never got my email last week reporting success, see below. You'd like me to test opening an archive on a read only Xsan volume?

That did the trick. I used the 1st setting and the 600GB proceeded at a
healthy clip, as high as 320MB/sec from one volume to another. The only odd
thing is it keeps capturing 3 empty folders - see attached. I also sent a
report.

It seems QRecall will not open an archive over a network share.. is that
correct?

Thanks
John

John Heagy wrote:I suppose you never got my email last week reporting success, see below.

I got your e-mail, I guess you didn't get my reply.

That did the trick. I used the 1st setting and the 600GB proceeded at a healthy clip, as high as 320MB/sec from one volume to another.

That's good to hear. Thanks for taking out the patched version out for a spin.

The only odd thing is it keeps capturing 3 empty folders - see attached. I also sent a report.

Thanks for the diagnostic report. I can't tell from the log what it's capturing or why. If you really want to find out, there's a special QRecall setting that will shed some light on the issue. See the QRLogCaptureDecisions setting. Once you've captured everything and nothing (ostensibly) has changed, set this option and preform another capture. It will log the reason it captured those three folders. Make sure you turn it off when you're done, or it will produce a tremendous amount of log file activity.

It seems QRecall will not open an archive over a network share.. is that correct?

Absolutely, you should be able to open it. QRecall should be able to open an archive on any readable volume.

Looking through your log it appears that you're getting an error -61 (write access error) when you try to open the archive. I surmise that the archive is on a volume or in a directory which is not writable by the client. This is, unfortunately, a side-effect of a workaround for a file sharing bug in AppleShare.

If you change the access privileges so that the archive is writable from the client (which is the norm) you shouldn't have any problems opening it. Note that archive must be writeable from the client to perform almost any other action (repair, compact, merge, delete, etc.).

In the mean time, I'll add the error -61 problem to the bug list.