QRecallDownloadIdentity KeysForumsSupport
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Groups] Back to home page 
Best way of capturing system and user data in two different archives  XML
Forum Index » Cookbook and FAQ
Author Message
Prion



Joined: 20-Nov-09 00:41
Messages: 18
Offline

Hi

I am somewhat new to this and would like to set up two different capture strategies that would
1) capture the entire system, preferrably overnight after which the system would be put to sleep or switched off
2) capture whatever might be worth capturing in terms of data in my user's space (Documents, Pictures, Movies, Music, etc.)

Ideally these would require little or no intervention to run after their initial setup. My main machine is a 15 inch MBP and I have two different external harddrives for the two tasks, for task #1 USB2 drive in an inclosure and for task #2 a Fw800 3.5 external HDD. Both are 500GB drives. In principle there is also a TimeCapsule 1TB that I am curently using for my TimeMachine Backup. The USB2 drive for the system capture contains a minimal Snow Leopard installation plus QRecall as the only program.

Question 1:
I imagine that capturing software while it is running is asking for trouble, so presumably I would need to be running a different operating system of the same type (10.6 in my case running on the 2.5 USB drive) to backup my main harddrive in the MBP. Likewise, I would need to be logged out for capturing my user data (task #2). Correct? How do I set this up then?
If so, this would limit my backup frequency in particular for task #2 because I would then let it run at noon and at night. Perhaps that is sufficient but I don't know. How do you handle this? I cannot imagine logging our every hour for my backup to take place.

Question 2:
Can I rename a drive that I previously captured data to with Qrecall and still continue capturing data to or restoring data from?
Can I copy the content of the 2.5 drive to another drive (with a different name) and use that drive to boot from? I would assume that I would need to use CarbonCopyCloner to keep the new drive bootable (or can I use Diskutil?).

Question 3:
Playing around with the TimeCapsule it seems as though QRecall is perfectly willing to accept an external harddrive connected to the TimeCapsule's USB port.
At least while I am at home (near the TCapsule) I would prefer to archive to that disk because then I would not need to connect and unconnect the drives used for backup.

Question 4:
I am unsure if I should include the user data also in the system backup to have some redundancy or if that just fills the thing up too quickly. Since I cannot easily prune the user data if I find that the case it might be worth asking you pros here.

Question 5: What do I need to do to ensure that my backup is safe on the one hand but that I will be able to restore a backup and not fail because of insufficient privileges over the files to be restored?

I know, many questions and I would appreciate your help very much

Prion
James Bucanek



Joined: 14-Feb-07 10:05
Messages: 1546
Online

Prion wrote:I am somewhat new to this and would like to set up two different capture strategies that would
1) capture the entire system, preferrably overnight after which the system would be put to sleep or switched off
2) capture whatever might be worth capturing in terms of data in my user's space (Documents, Pictures, Movies, Music, etc.)

This should be easy, and you have a variety of choices on how you implement it.

But first, I'll answer your questions.

Question 1:
I imagine that capturing software while it is running is asking for trouble, so presumably I would need to be running a different operating system of the same type (10.6 in my case running on the 2.5 USB drive) to backup my main harddrive in the MBP. Likewise, I would need to be logged out for capturing my user data (task #2). Correct? How do I set this up then?
If so, this would limit my backup frequency in particular for task #2 because I would then let it run at noon and at night. Perhaps that is sufficient but I don't know. How do you handle this? I cannot imagine logging our every hour for my backup to take place.

Not at all. QRecall is perfectly capable of capturing your system while booted from that volume, and your home folder while logged in.

There are potential problems, but they're with specific applications. The usual suspects are Mail, Aperture, VMWare, Parallels, etc. These applications keep data cached in memory; capturing their data files create an incomplete picture of their state, that may possibly be invalid when restored. The solution is to quite these applications before starting a capture, but you don't have to log out. (Note that this particular problem applies to any backup solution, not just QRecall.)

If you really want to make sure your home folder is captured in a pristine state, you can capture it while logged out. QRecall can do this. You can even schedule a capture action to occur when you log out. If you regularly log out of your account after working, this might be your best solution.

Question 2:
Can I rename a drive that I previously captured data to with Qrecall and still continue capturing data to or restoring data from?

Yes. You may need to point the actions at the archive after you've renamed or moved it. To do this, open up any action that uses that archive and select the archive again.

Can I copy the content of the 2.5 drive to another drive (with a different name) and use that drive to boot from? I would assume that I would need to use CarbonCopyCloner to keep the new drive bootable (or can I use Diskutil?).

Yes. I've never used Disk Utility to clone a drive, but basically whatever results in a bootable drive will work. The QRecall archive is just a document. You can move, copy, or rename it as you would any document.

Question 3:
Playing around with the TimeCapsule it seems as though QRecall is perfectly willing to accept an external harddrive connected to the TimeCapsule's USB port.

QRecall is happy to use a QRecall archive on any mountable volume.

At least while I am at home (near the TCapsule) I would prefer to archive to that disk because then I would not need to connect and unconnect the drives used for backup.

I think that's the best solution, as I'll illustrate later. The less you have to remember about your backups the better.

Question 4:
I am unsure if I should include the user data also in the system backup to have some redundancy or if that just fills the thing up too quickly.

That's entirely up to you.

Since I cannot easily prune the user data if I find that the case it might be worth asking you pros here.

You can very easily prune the data. You can open an archive and completely delete any previously captured file or folder. So you can start by capturing your entire boot volume to your USB drive. If the drive begins to fill up too quickly (and I seriously doubt it will), you can delete the /Users folder from your archive and then exclude it from future captures by adding it to the "exclude" list in the capture action.

The advantages of capturing everything are convenience and redundancy. If you exclude you're /Users folder from the whole-volume capture, then when the need arrises to restore your boot volume you'll have to do it in two stages: first from the "system" archive and then your user data from your "user" archive. And you won't be able to boot until you've done both. Not a big deal, but worth noting.

The other advantage is redundancy. Having two backups is twice the security.

Question 5: What do I need to do to ensure that my backup is safe on the one hand but that I will be able to restore a backup and not fail because of insufficient privileges over the files to be restored?

To verify the integrity of your QRecall archive, create an action the periodically verifies the archive. I would suggest scheduling this to run once a week. I run them almost daily, but I'm paranoid and I'm always fiddling with the code, so my archives are more likely to get messed up more than yours.

When you install QRecall, you'll want to pre-authorize QRecall to use administrative privileges. This allows QRecall to run as the superuser (root), which gives it complete authority to capture and restore every aspect of your filesystem. When you restore from your emergency drive, you'll either want to pre-authorize QRecall or manually authorize it once to perform the restore.

I know, many questions and I would appreciate your help very much

Here's my suggestion:

1) Creata a capture strategy to capture your entire boot volume to your USB drive automatically when the drive is connected. The capture assistant will set this up for you (Help > Capture Assistent, choose your boot volume, the archive on the USB drive, tell the assistent it's on a removable volume and you want to capture when it connects).

Your daily backup routine: when you're done with work, quite your major applications and plug in the USB drive. The capture will start immediately.

2) Create a capture strategy to capture your home folder, or /Users folder, to your networked Time Capture volume. Again, use the capture assistent to set this up as before, but tell it to run at a particular time each day. After the capture assistent has created the actions, open the Actions window and change the schedule of the capture action to run more often (like every 4 hours), or maybe create duplicates that run at different times during the day (one at noon, another at 7PM, etc.).

Your daily backup routine: make sure your Time Capture volume is mounted at the times when the capture is scheduled to run.

If you want QRecall to remind you of these things, consider the following changes:

Change the schedule of the USB drive capture to a specific time and add a Hold While No Archive condition. QRecall will start the capture every day, where it will appear in your QRecall Activity window, and wait patiently for you to plug in your USB drive.

Remove the Ignore If No Archive conditional from the hourly capture of your home folder. The action will run regardless of whether your network volume is mounted or not. If the volume can be mounted, QRecall will mount it and run. It the volume isn't available, QRecall will complain.

Install Growl.

I hope that all makes sense. Write again if you have more questions.
[Email]
James Bucanek



Joined: 14-Feb-07 10:05
Messages: 1546
Online

One last thought:

If you capture your home folder to both your USB drive and network volume, set up the merge schedule to you keep frequent layers on your network volumes (like every capture for the past week, then daily for 60 days or so), but only keep daily and weekly layers in your USB volume (say, daily for two weeks, weekly for two months, then monthly for a couple of years.)

This will keep both of your archives lean and mean, but still preserve a lot of near-term versions.

- QRecall Development -
[Email]
Ben Bass



Joined: 17-Dec-12 16:20
Messages: 3
Offline

James Bucanek wrote:
Remove the Ignore If No Archive conditional from the hourly capture of your home folder. The action will run regardless of whether your network volume is mounted or not. If the volume can be mounted, QRecall will mount it and run. It the volume isn't available, QRecall will complain.


Hi there.

I was wondering if there was a way to keep the QRecall archive on a network volume, have it auto-mount when the volume is available, but not have QRecall or the OS complain that the network volume is unreachable. I have many clients that have laptops and I don't want to have an error dialogue pop up when they aren't on the network, I would like it to just fail gracefully.

I saw some early comments about a command line version, which would be awesome as I could write my own scripts if necessary where I can check which network they are on and have the archiving cleanly fail if they are not.

Thank you!

-Ben Bass
James Bucanek



Joined: 14-Feb-07 10:05
Messages: 1546
Online

Ben Bass wrote:I was wondering if there was a way to keep the QRecall archive on a network volume, have it auto-mount when the volume is available, but not have QRecall or the OS complain that the network volume is unreachable.

Not at this time. QRecall will attempt to auto-mount any volume before an action begins, but a failure to mount that volume is considered a fatal error, and QRecall will complain about it.

If this is something that people would like, I can put it on the to-do list.

I saw some early comments about a command line version, which would be awesome as I could write my own scripts if necessary where I can check which network they are on and have the archiving cleanly fail if they are not.

A command line version is definitely in the works. Right now, QRecall is undergoing a lot of testing. It just had its core file system logic completely rewritten so it uses Apple's approved APIs. It's going to need a lot of testing before I spring it on any unexacting beta testers.
[Email]
Ben Bass



Joined: 17-Dec-12 16:20
Messages: 3
Offline

James Bucanek wrote:
Ben Bass wrote:I was wondering if there was a way to keep the QRecall archive on a network volume, have it auto-mount when the volume is available, but not have QRecall or the OS complain that the network volume is unreachable.

Not at this time. QRecall will attempt to auto-mount any volume before an action begins, but a failure to mount that volume is considered a fatal error, and QRecall will complain about it.

If this is something that people would like, I can put it on the to-do list.

I saw some early comments about a command line version, which would be awesome as I could write my own scripts if necessary where I can check which network they are on and have the archiving cleanly fail if they are not.

A command line version is definitely in the works. Right now, QRecall is undergoing a lot of testing. It just had its core file system logic completely rewritten so it uses Apple's approved APIs. It's going to need a lot of testing before I spring it on any unexacting beta testers.


Having an option to have QRecall not complain when it can't mount a volume would be awesome. Glad to hear there still is a command line version in the works.

Thank you!
Ben Bass



Joined: 17-Dec-12 16:20
Messages: 3
Offline

James Bucanek wrote:
Ben Bass wrote:I was wondering if there was a way to keep the QRecall archive on a network volume, have it auto-mount when the volume is available, but not have QRecall or the OS complain that the network volume is unreachable.

Not at this time. QRecall will attempt to auto-mount any volume before an action begins, but a failure to mount that volume is considered a fatal error, and QRecall will complain about it.

If this is something that people would like, I can put it on the to-do list.

I saw some early comments about a command line version, which would be awesome as I could write my own scripts if necessary where I can check which network they are on and have the archiving cleanly fail if they are not.

A command line version is definitely in the works. Right now, QRecall is undergoing a lot of testing. It just had its core file system logic completely rewritten so it uses Apple's approved APIs. It's going to need a lot of testing before I spring it on any unexacting beta testers.


Just to update you. I setup an automount on my mac by editing my /etc/auto_master and placing the QRecall archive on the network volume mounted via the automount. When I am on the network it works like a charm, and with the "Ignore Archive" conditional set it quietly fails, which is the behavior I was going for. I am trying it out now to see how well it works over VPN's. I might deal with this through time of day conditionals, but will see what I come across.
 
Forum Index » Cookbook and FAQ
Go to:   
Powered by JForum 2.1.8 © JForum Team