QRecall Community Forum
  [Search] Search   [Recent Topics] Recent Topics   [Hottest Topics] Hottest Topics   [Top Downloads] Top Downloads   [Groups] Back to home page 
[Register] Register /  [Login] Login 

Destination file in a server? RSS feed
Forum Index » General
Author Message
Paulino Michelazzo


Joined: Mar 7, 2015
Messages: 4
Offline
Hi people,

Looking for an alternative to my backup tool with less space use, I found Qrecall.

Today I have two servers in different datacentes to store the backups of my files. Is this possible to make the same with Qrecall? I install the program but I can't find this kind of option (FTP, SFTP, WEbDav, etc)

Any clues?

Thanks!
Paulino
James Bucanek


Joined: Feb 14, 2007
Messages: 1572
Offline
Paulino,

The current version of QRecall can only capture to an archive document on a mounted filesystem. If you can mount your server's backup volume as a remote volume on your client system, QRecall can work that way. QRecall understands remote volumes and external drives and will automatically attempt to mount the archive volume when an action starts.

Another possible solution (which is one I use) is to capture to a local external volume and then periodically rsync that archive with a duplicate on your server(s).

There are long-term plans for QRecall to provide true client/server backup services, but that's a ways off?there are so many new features to implement first.

- QRecall Development -
[Email]
Paulino Michelazzo


Joined: Mar 7, 2015
Messages: 4
Offline
Hi James,

Thank you for your fast answer.

I don't have problem to make a rsync between some shared folder and my servers but I would like to know what's the approach using the Qrecall on this way. I mean, if I make a sync of the Qrecall archive (that I put in a Linux NAS), which kind of data the rsync send to the server? Just the different bytes between files or all versions?

Is this transparent?

Thanks again.
James Bucanek


Joined: Feb 14, 2007
Messages: 1572
Offline
Paulino,

Essentially, you'll use QRecall to make incremental backup to a local drive and then periodically mirror your archive (via rsync) to your server so you have a backup of your backup.

Here's what I do:
  • Schedule QRecall to make regular captures to an archive on an external volume.

  • Added conditions to the actions so I'm sure that no actions are modifying the archive during a regular period each day (say, between 4:00 and 7:00 AM).

  • Once or twice a week, run an rsync that mirrors my active archive with its remote copy. I do this starting at 4:00 AM so that the archive isn't changing while the rsync occurs.


  • QRecall will capture and de-duplicate your files into the local archive. The rsync will then transfer just those changes to the remote archive to keep it up-to-date.

    This is sub-optimal solution for a couple of reasons. First, it requires you maintain two copies of your backup archive, one local and one remote. (This can also be an advantage if that's what you want.) The other problem is that rsync makes a local copy of the archive being updated, so the remote copy of the archive has to be a volume with enough space to duplicate it every time it's synced.

    I've been running this solution on my server for about 5 years. Actually, I do it in reverse, keeping a copy of my server's backup on my local computer, but the principle is the same. As long as the archive is a reasonable size (say 200-300GB), it's very manageable.

    - QRecall Development -
    [Email]
    Paulino Michelazzo


    Joined: Mar 7, 2015
    Messages: 4
    Offline
    Hi James,

    My scenario is:

    - 800Gb of photos (growing around 20~30Gb/month)
    - 150Gb musics (growing around 3~5Gb/month)
    - 400GB videos (growing around 30~50Gb/month)

    These files are stored inside a Mac Mini with a 2Gb internal HDD synced with my NAS.

    Today I have two servers with similar specs where the discs in both are 7x1.5Tb in RAID 5. Also I have a NAS in my local network with 8Tb (JBOD).

    The problem remains with the program that make the backup to my servers. For example: if I change one single bit in a photo (e.g. change the saturation), it's make a backup of the whole archive. So, if I edit 100 photos/day, can you imagine how many space I need to buy no?

    I think that your approach could be fine but I would like to be sure about how many data I need to backup in each cycle. I can buy more disks for the Mini (actually I'm buying a thunderbolt NAS) to support the growing of my files and this is not my problem (also I can put more drives in my servers). The real problem is the hours that the system spend to make the bakup for the servers. Sometimes, in a hard day of work, the backup need two days for a cycle just because I change small parts of videos (cut 2 seconds for example) or change the photos. That's a mess!

    Regards and tanks again for your tips.

    James Bucanek


    Joined: Feb 14, 2007
    Messages: 1572
    Offline
    Paulino Michelazzo wrote:I think that your approach could be fine but I would like to be sure about how many data I need to backup in each cycle.

    This is where trial identity keys are useful.

    Get a trial key, install QRecall, and just try it out. Start with your music or just a subset of your photos/videos and see how much QRecall uses and how much it adds each day. (Tip: the log details will record exactly how many files it captured, how much duplicate data was discovered, and how much the archive grew.)

    Here are some things to consider:

  • QRecall performs block level data de-duplication. If editing a photo only changes "a few bytes," QRecall will find the one block of data that was changed and will only capture that single block.

  • QRecall can also perform shifted quanta detection to locate duplicate blocks that have moved to a different offset in the file. If editing a photo or movie only modifies or appends metadata, but doesn't move any other data around, then block level de-duplication is fine. But if you're using a program like Photoshop that rewrites the entire file, a lot of the data gets rewritten with different offsets. For example, open an image, add a filter layer, and save the file. The original image data is still there, but gets written at a different offset. Turning on QRecall's shifted quanta detection can find that relocated duplicate data. (Note: shifted quanta detection is much more CPU and I/O intensive, but sometimes it's worth it.)

  • QRecall lets you choose what you capture, when, and to where. You can set up different archives for your photos, music, and videos (since it's unlikely that there is any duplicate data shared between them). This will also let you experiment with different shifted quanta detection levels, compression levels, etc., to find the settings the work best for each category of data.

  • Dividing up your backup into multiple archives also means that using rsync to mirror them to a remote server will be much more manageable and will require less free disk space.

  • - QRecall Development -
    [Email]
    Paulino Michelazzo


    Joined: Mar 7, 2015
    Messages: 4
    Offline
    Hi James,

    Thanks for the infos. The log part is what I'm looking for.

    Let me try some days. If Qrecall work as expected, you have a new client

    Regards,
     
    Forum Index » General
    Go to:   
    Mobile view
    Powered by JForum 2.8.2 © 2022 JForum Team • Maintained by Andowson Chang and Ulf Dittmer