Docstoc

Backup using OpenSource utilities

Document Sample
Backup using OpenSource utilities Powered By Docstoc
					       Backup using OpenSource utilities
       Daniel Steiner . Consultant . 24. Oct. 2008



 1      Preface
This document presents an overview of the available Open Source backup softwares. It will show several
possibilities and alternative, in order to give you a feeling about the power of these Open Source solutions.

If you search for backup software on http://www.sourceforge.net, you get more than 700 results. This
demonstrates the large range of Open Source software for backup and recover purposes. The scope of this
document will of course be reduced and focuses on the most interesting softwares.

Most OpenSource backup utilities are based on Linux / Unix standard tools like tar, cpio, bzip2, gzip, ssh, rsync,
mt and mtx. Therefore an easy installation is always possible, indeed all these tools are part of Linux
distributions or other UNIX derivatives. This document describes only a few, but very useful applications.

The range of software varies between Enterprise level and very simple software applications. The chapter,
Software list, presents some important Open Source backup projects. From this list, We picked a few
applications to describe them more in deep:

        BackupPC:                             Network backup server, which writes the backups to disk only

        Bacula:                               Network backup server, which can handle tapes and tape
                                              libraries

        Monderescue:                          Disaster recovery tool, which can create directly bootable
                                              medias

        SystemRescueCD:                       Bootable CD, which contains a lot of tools for backup and
                                              recover procedures

        Installserver based recovery:         It's also possible to restore using an installation system

        Trivadis backup module:               Perl script which collects partition, LVM and software raid
                                              information and creates a partimage or compressed tar archive

        rdiff-backup:                         Mirror application based on rsync

        rsnapshot:                            Mirror application based on rsync




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 1 / 34
 2        Software list

2.1 Data backup
Below a not complete list of OpenSource backup utilities. Yellow marked tools are presented in this document:

      Software                                     Web URL                                     Notes
BackupPC:            http://backuppc.sourceforge.net/info.html
Amanda:              http://amanda.zmanda.com/
Bacula:              http://www.bacula.org/
Cedar Backup:        http://cedar-backup.sourceforge.net/
Cobian Backup:       http://www.educ.umu.se/~cobian/cobianbackup.htm                  Windows only! Delphi
rsnapshot:           http://www.rsnapshot.org/
rdiff-backup:        http://www.nongnu.org/rdiff-backup/
snapbackup:          http://snapbackup.com/
lbackup:             http://connect.homeunix.com/lbackup/doku.php?id=overview
synbak:              http://www.initzero.it/products/opensource/synbak/
Restore:             http://restore-backup.com/



2.2 Cloning or disaster:
Also not complete list of cloning or disaster OpenSource software:

         Software                                         Web URL                                   Notes
CDBkup:                  http://cdbkup.sourceforge.net/
partimage:               http://www.partimage.org/Main_Page
SysrescuCD:              http://www.sysresccd.org/Main_Page
Clonezilla:              http://clonezilla.sourceforge.net/
CloneIt:                 http://www.ferzkopp.net/joomla/content/view/12/14/
FOG (Freee Ghost):       http://freeghost.no-ip.org/index.php?option=com_frontpage&Itemid=1
Mondoarchive:            http://www.mondorescue.org/, ftp://ftp.mondorescue.org/
snapbackup:              http://snapbackup.com/



2.3 Network mirroring software
To mirror over network or locally. This list also not complete, shows only Open Source software:

         Software                                         Web URL                                   Notes
Unison:                  http://www.cis.upenn.edu/~bcpierce/unison/index.html
Dirvish:                 http://www.dirvish.org/
rsync:                   http://www.samba.org/rsync/




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 2 / 34
 3      Backup scenarios
To understand which backup application can be used for which requirement, a short presentation of the
different backup scenarios are shown in this chapter. The backup differs a lot depending on the requirements of
the service level agreements. Sometimes, it's enough to backup user and application data only. But, if the
availability of a service must be guaranteed, a short restore time is required also for the operating system
(Recovery Time Objective).


3.1 Data backups
Only user, OS configuration and application data is saved. This backups can be done periodically (scheduled).




              Picture 1: Data backups


3.2 Operating system backups
All operating system relevant data, like boot parameters or kernel relevant setting, are part of this backup type.
Those backups can be part of data backups, so you also can schedule them.




               Picture 2: Operating system backups




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 3 / 34
3.3 Data restores
Data restore means, the Operating system has already been restored. If the backup contains operating system
data, it can only be restored, after the operating system itself has been restored using disaster recovery software,
see next chapter.




                                                                                            Pict
               ure 3: Data restores



3.4 Operating system or disaster recoveries
If you have to recover a crashed server, for example loss of all data on operating system disk or hardware
damage, you will have to follow this scenario. The procedure to recover such a server is usually divided in
several steps:

    1.   If needed, installation of new hardware
    2.   Installation of a mini operating system
    3.   Recovering of operating system relevant data
    4.   Recovering of user and application data




                                                                                            Pi
          cture 4: Operating system or disaster recoveries




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 4 / 34
 4      BackupPC




       Picture 5: BackupPC, exemplary

BackupPC is a high-performance, enterprise-grade system for backing up Linux, Windows PC's and laptops to a
server's disk. BackupPC is highly configurable and easy to install and maintain. It can be used for daily, weekly
or monthly periodic backups.

Because the costs of disks and raid systems, it is now practical and cost effective to backup servers directly onto
local storage on a backup server. But, if you want to save your data for a long time, you should plan to backup
these data to tapes. BackupPC provides you a very reliable, flexible backup solution, if you don't need to
backup the data on tapes. If you need to backup the saved data on tape you have a lot of choices, if you want to
use OpenSource software.


Category                         Data backups
Tools used                       rsync, samba, ssh, nfs or tar
Gui                              Web based (Apache / perl)
Requirements                     Apache, Perl, rsnyc, samba, tar, gzip, bzip2, ...
Schedule able                    Yes, internal
Configuration                    /etc/BackupPC, or Webgui




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 5 / 34
4.1 Features
        A powerful web (HTTP/CGI) user interface allows administrators to view log files, configuration, current
        status and allows users to initiate and cancel backups, browse and restore files from backups
        Different backup mechanism are available (tar, scp, rsync, ...)
        Incremental backups possible
        A clever pooling scheme minimizes disk storage and disk I/O. Identical files across multiple backups of
        the same or different PCs are stored only once resulting in substantial savings in disk storage and disk
        I/O
        Optional compression support further reducing disk storage. Since only new files (not already pooled)
        need to be compressed, there is only a modest impact on CPU time
        Supports mobile environments where laptops are only intermittently connected to the network and have
        dynamic IP addresses (DHCP)
        Flexible configuration parameters allow multiple backups to be performed in parallel, specification of
        which shares to backup, which directories to backup or not backup, various schedules for full and
        incremental backups, schedules for email reminders to users and so on. Configuration parameters can
        be set system-wide or also on a per-PC basis
        Users are sent periodic email reminders if their PC has not recently been backed up. Email content,
        timing and policies are configurable
        Tested on Linux, Freenix and Solaris hosts and Linux, Win95, Win98, Win2000, WinXP clients.
        Detailed documentation


4.2 Advantages
        No client-side software is needed. The standard smb protocol is used to extract backup data on
        Windows clients. On Linux clients, tar over ssh/rsh/nfs is used to backup the data.
        Easy to use web interface
        Good backup status informations in the web interface
        A full set of restore options is supported, including direct restore (via smbclient, tar, or rsync/rsyncd) or
        downloading a zip or tar file


4.3 Disadvantages
        No tape backup is possible
        Setup of UNIX clients needs UNIX and SSH knowledge
        Backup storage cannot be on different file systems, because of using hard links
        Uses quite a lot of CPU resources (rsync)


4.4 Backup
Usually backups are executed in specified time slots, but, if a server was not reachable for a longer time period,
the backup is started immediately after BackupPC recognizes the client in the network. Using BackupPC, you do
not schedule the backup at a specified time, you can define a time slot, when the backup should be usually
done. Following variables control the backup time slots:

        $Conf{WakeupSchedule}: In which time period should be the backup started? Can be defined generally
        or by individual clients.
        $Conf{BlackoutPeriods}: What time of day without a normal backup. Normally defined generally, but
        you can define this also for individual clients.




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 6 / 34
Status of backups can be reviewed in the web interface:




  Picture 6: Overview of backups in BackupPC webinterface


It's also possible to start backups manually in the web interface. Just select the client and click on the
corresponding backup button (Start Incr Backup or Start Full Backup), see Picture 7 next.




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 7 / 34
4.5 Recovery
The restore is also quite easy, like the backup. Everything can be done in the web interface.

        Select date of the backup to restore, just click on the Backup# link to select it:




                                                                                                          Pi
        cture 7: Available backup for restore in BackupPC webinterface

        Select data (files and folders) to restore:




        Picture 8: Select files and folder to restore in BackupPC




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 8 / 34
   Restore it:




   Picture 9: Restore options in BackupPC


   For example a resulting Tar archive looks like this, tar tf ...:
   tar tf restore.gtar
   ./apache2/
   ./apache2/conf.d/
   ./apache2/conf.d/inst_server.conf.in




info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 9 / 34
 5       Bacula




            Picture 10: Bacula, exemplary

Bacula provides the system administrator with a robust, well documented and seriously maintained backup
solution comparable to “enterprise” backup frameworks. The backup catalog is stored in a Database. Bacula is a
set of programs which allow the system administrator to manage backup, recover, and verify backup sets across
a network of computers of different kinds. Bacula can also run entirely upon a single computer and can backup
to various types of media, including tape and disk.
In technical terms, it is a network Client/Server based backup program. Bacula is quite easy to use and efficient.
For instance by offering many advanced storage management features that make it easy to find and recover lost
or damaged files. Due to its modular design, Bacula is scalable from small single computer systems to systems
composed of hundreds of computers located over a large network.

Category                         Data backups
Tools used                       Programm Language         C++
Gui                              QT4 Gui, console based Gui
Requirements                     MySQL or Postgress DB, mtx for librarys
Schedule able                    Internal scheduler for automatic and simultaneous job execution
Configuration                    Internal configuration files




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 10 / 34
5.1 Features
     A central server and catalog with distributed backup
     All components communicate via the network and are deployed separately.
     Internal scheduler for automatic and simultaneous job execution with priorities.
     Interactive restore of one or more files from:
          current backup
          prior backup of time and date
          list of files/directories to restore
          restore by JobId
           ...
     Simple administration with consoles (command line, GUI, and web)
     Labeled Volumes, to prevent accidental overwriting
     Support for ANSI / IBM labels
     Machine independent Volume data format - extensible
     Support for Unicode on Win32; UTF-8 on Unix
     Python interpreter for user “event” (job start, end, ...) scripting
     Rescue CDROM for “bare metal” recovery.
     Hardware Features
          Backups can span multiple volumes
          Multiple backups (jobs, clients, OSes) per volume
          Supports most tape drives with configurable Device resources
          Support for multiple drive autochangers (libraries)
          Supports tape barcode readers
          Extensive Pool and Volume library management
          Rapid restoration of individual files (one user reported 4 to 6
          hours with tar and 3 to 4 minutes with Bacula!)
     Security Features
          Daemon authorization with CRAM-MD5
          Director and Storage daemon can be run non-root
          MD5, SHA1, ... signatures for each file
          CRC checksum for each Volume block
          Restricted consoles and tray-monitors
          Communications (TLS) encryption
          Data (PKI) encryption
          Tripwire like intrusion detection (Verify)


5.2 Advantages
     very good support of platforms, operating systems and hardware
     functionality comparable to enterprise backup solutions
     robust
     easy to use
     excellent documentation
     no license costs


5.3 Disadvantages
     initial configuration not easy

5.4 Backup
     scheduler for automatic and simultaneous job execution




  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 11 / 34
Picture 11: Bacula scheduler in action


5.5 Recovery
        fast recovery possible through the GUI and informations from the catalog

For more detailed information about Bacula, please consult the Bacula documentation of Roman Gächter on
http://www.trivadis.com/de/know-how-community/download-area.html




    info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 12 / 34
 6       Mondorescue




                   Picture 12: Mondorescue, exemplary

Mondorescue is a disaster recovery solution. It supports Linux (i386, x86_64, ia64) and FreeBSD (i386). It's
packaged for multiple distributions (RedHat, RHEL, SuSE, SLES, Mandriva, Debian, Gentoo).

It supports tapes, disks, network and CD/DVD as backup media, multiple filesystems, LVM, software and
hardware Raid.

Category                        Baremetal backups (Disaster)
Tools used                      mkisofs, cdrecord, syslinux, gzip, bzip2, ...
Gui                             ncurses based gui (command line)
Requirements                    CD writing tools, syslinux, mindi, gzip, bzip2, ...
Schedule able                   Yes, via cron
Configuration                   none




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 13 / 34
6.1 Features
        Automatic system rescue backup through crontab
        The images can be created directly on running servers
        The backup can be written directly to CD, DVD, local disk or network drive
        Created boot image uses the same kernel as the original system, so you shouldn't have problems with
        drivers
        Interactive restore mode, this is very useful for restores on different systems
        Multiple Linux distributions supported


6.2 Advantages
        Very easy to handle backup procedure
        Could be used to clone servers
        Differential backups are possible


6.3 Disadvantages
        Sometimes, it does not properly restore without using interactive mode, if you have complex disk
        configuration like LVM or software RAID setups
        Not all messages a are coherent
        Restore is not that simple like a backup


6.4 Backup
To backup, you can choose between two possibilities:

    1. Interactive backup with a ncurses based gui (only in console!)
    2. Command line, you must enter all needed options as parameters in the command line!

Very important: To make sure, the restore is working fine, you must test it, a wrong backup parameter can
make the restore unusable!




    info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 14 / 34
 6.4.1 ............................................................................................................................ Inter
active backup
For an interactive backup, just type mondoarchive:

 Backup command, mondoarchive ...:
 mondoarchive
 Initializing...
 See /var/log/mondoarchive.log for details of backup run.
 Checking sanity of your Linux distribution
 /sbin/mkfs.vfat
 Filesystem        1048576-blocks     Used Available Capacity Mounted on
 /dev/sda1            90036    4577 80886          6% /
 Done.
 Execution run ended; result=1
 Type 'less /var/log/mondoarchive.log' to see the output log


Following steps and questions will occur during interactive backup:

          Welcome screen, choose the media for the backup:




          Picture 13: Mondorescue backup media options


          Where to store the ISO image?

          Which compression level?

          What's the maximum size of DVD or CDROM?

          What name prefix for the backup archive?

          What's should be included in backup?

          What should be excluded from backup?

          Should the written data be verified?




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 15 / 34
   Picture 14: Mondorescue, catalog                     Picture 15: Mondorescue, boot and data disk




   Picture 16: Mondorescue, filesystem backup           Picture 17: Mondorescue, filesystem backup
                                                        progress
   Execution?

   Floppy disk creation always fails, but you can disable the creation using -F option?

   Backup finished!




info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 16 / 34
 6.4.2 ............................................................................................................................ Non
interactive backup
To start the non interactive backup, you must only execute one command. Depending the options you have
given, the backup will be done direct to CDROM, DVD, NFS share or local disk. mondoarchive creates in all
cases a bootable Disk, which can be booted for restore action. For further details about using mondoarchive,
please refer to the manual pages.

 Backup command, mondoarchive ...:
 mondoarchive -O -p ubuntu-t61 -i -N -E /backup -d /backup/ -s 4000m -F -G -l GRUB
 Initializing...
 See /var/log/mondoarchive.log for details of backup run.
 Checking sanity of your Linux distribution
 Disk /dev/dm-0 doesn't contain a valid partition table
 Disk /dev/dm-1 doesn't contain a valid partition table
 Disk /dev/dm-2 doesn't contain a valid partition table
 Disk /dev/dm-3 doesn't contain a valid partition table
 Disk /dev/dm-4 doesn't contain a valid partition table
 Disk /dev/dm-5 doesn't contain a valid partition table
 Disk /dev/dm-6 doesn't contain a valid partition table
 Done.
 BusyBox's sources are available from http://www.busybox.net
 Making catalog of files to be backed up
 ---evalcall---1---    Making catalog of /

 ...

 ---evalcall---E---
 Call to mkisofs to make ISO (ISO #1) ...OK
 Done.
 Done.
 Writing boot+data floppy images to disk
 No Imgs
 ---promptpopup---1--- No regular Boot+data floppies were created due of space constraints. However, you can burn
 /var/cache/mindi/mondorescue.iso to a CD and boot from that.
 ---promptpopup---Q--- [OK] ---
 --> Backup and/or verify ran to completion. However, errors did occur.
 /var/cache/mindi/mondorescue.iso, a boot/utility CD, is available if you want it
 Data archived OK.
 See /var/log/mondoarchive.log for details of backup run.




 6.5 Recovery
Very Important: You must take care about the following criteria's.

          If you have LVM partitions, the nuke (full automatic) restore does not work properly! You need to
          partition the first hard disk manually.

          After partitioning, you need to reboot the server, before you can continue with the restore process. To
          do so, you need to make a configuration save of the hard disk. Then you have to restore the server using
          mondo. You must always use interactive mode to restore such a server.

          Additionally during interactive restore mode, you must enter the configurations for /boot and swap
          partitions. Just enter /boot without any size value, for the swap partition you must enter in the mount
          point and also for the file system type setting “swap”.

          After this you can restore the server. If you will be asked for erasing the disk, enter “No” to keep the
          disk partitions.

          On Ubuntu installations, you must edit the /etc/fstab and the grub configuration files to correct the disk
          label information for all file systems. After the restore the UID's of the file systems are different.




       info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 17 / 34
 7       System rescue CD




            Picture 18: System rescue CD, exemplary

SystemRescueCd is a Linux (Gentoo) system on a bootable CD-ROM which contains tools to recover or repair
your system. It aims to provide an easy way to carry out admin tasks on your computer, such as creating and
editing the partitions of the hard disk. It contains a lot of system utilities (parted, partimage, fstools, ...) and basic
tools (editors, midnight commander, network tools). It is very easy to use: Just boot the CDROM. The kernel
supports most of the important file systems (ext2/ext3, reiserfs, reiser4, xfs, jfs, vfat, ntfs, iso9660), as well as
network filesystems (samba and nfs).

Category                           Recovery CD
Tools used                         Gentoo Linux based bootable CD
Gui                                X11 is available
Requirements                       CD burner
Schedule able                      -
Configuration                      none




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 18 / 34
7.1 Features
     Boot able recovery environment
     Includes all necessary tools to recover or repair a server
     Also usable for Windows repair actions
     Includes graphical tools like gparted, browser, ...
     From version 1.0 on, chroot is now possible on 64bit servers


7.2 Advantages
     Very flexible universal boot CDROM
     X11 window based applications
     PXE (network based booting via DHCP and TFTP server) network boot possible for versions above or
     equal 1.0
     Supports a lot of hardware and file system types


7.3 Disadvantages
     Requires UNIX skills, because the most commands are command line based
     Not bootable over network without significant changes of the ISO image for versions below 1.0




  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 19 / 34
7.4 Usage
Just insert the CDROM into drive and boot from it. When boot prompt is shown, you can switch between
different help screens using <CRTL> + <ALT> + F1, F2, F3, F4, F5, F6 and F7 keys.




    Picture 19: System rescue CD boot welcome screen

It's possible to start X11 window session, to do so, start the wizard command. This lets you choose the type of
X11 configuration. To make sure the X11 server starts, choose Xvesa-cfg mode, in this mode you can select the
screen resolution. This mode is only available on 32-bit kernels.




    Picture 20: System rescue CD application examples




    info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 20 / 34
 8       Installserver based recovery




            Picture 21: Installserver based recover, exemplary

An installserver is a service which provides the ability to auto install any client over network. Most Linux
support the automated installation of operating system. Trivadis recommends an installation through
installserver, if this possible. An installserver consists of the following standard network services:
          DHCP server
          TFTP server
          DNS server
          NFS, FTP, HTTP or HTTPS server
          Any automatic installation mechanism, this is usually provided by the Linux distributions:
          For Redhat based distributions:              Kickstart
          For SUSE based distributions:                Autoyast
          For Debian based distributions:              Autoinstall (FAI)

To recover a server which was confronted with a disaster/failure (no hard disk or no data is accessible), you have
to install a new operating system, before you can restore any backup data. This is required, if you didn't perform
any disaster backup previously. In case, your installation was already an install server based installation, you can
install this system using the same autoyast or kickstart configuration file.

An automated recovery of the operating system is also possible, even if you didn't install the failed system from
installserver. But, in this case, you must collect the necessary configuration data previously from this system, see
Picture 22.
Note: This scenario does not include any backup functionality, it only helps to reactivate the operating system as
fast as possible. In case of required restore, you must use any backup software.

Category                          Bare metal backups (Disaster)
Tools used                        Kickstart, Autoyast, ...
Gui                               none (Satellite server is web based)
Requirements                      Linux server, Apache, FTP server, TFTP server, DHCP server, DNS server, ...
Schedule able                     no
Configuration                     Kickstart or Autoyast files



      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 21 / 34
8.1 Features
     Fast recovery option, an installation takes about 20 minutes
     Customizable using scripts


8.2 Advantages
     Very fast OS recovering
     Flexible to restore on different architectures
     Can be used for training or test center environments


8.3 Disadvantages
     Scripting knowledge to make customized installations
     Sometimes difficult because of PXE (network based booting via DHCP and TFTP server) booting




  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 22 / 34
 8.4 Usage


 8.4.1 ............................................................................................................................ Colle
cting configurations
If no kickstart or autoyast profile is available, you must create a new one from a template. For this, you need the
configuration of the server, you have to restore. You can collect this data using cfg2html, the “Trivadis
Installserver documentation” or you should have collected the data manually previously to the server crash. The
cfg2html package is available from updates or install server.

 8.4.2 ............................................................................................................................ Auto
yast and kickstart
The next diagram shows an overview for the install server based recovery procedure:




        Picture 22: Installserver based recovery overview

Prerequisites:

          Normal daily backups of user data and operating system configurations
          Periodic snapshots of system configuration, for example with cfg2html
          Optionally, a script which collects all necessary data to build a autoyast or kickstart file

Backup environment:

          Backup server which backups user and OS data daily (Bacula, BackupPC)
          Configuration database which contains the collected configuration data (Web server)

Installserver:

          Trivadis install server or similar
          Configuration database with all collected autoyast and kickstart files




     info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 23 / 34
 9       Trivadis backup module
                                                                                                 The
                                                                                                 Trivadis
                                                                                                 backup
                                                                                                 module
                                                                                                 provides an
                                                                                                 easy perl-
                                                                                                 based
                                                                                                 solution to
                                                                                                 perform
                                                                                                 backups
                                                                                                 from a
                                                                                                 running
                                                                                                 server. The
                                                                                                 script could
                                                                                                 also be
                                                                                                 used for full
                                                                                                 backups, in
                                                                                                 this case
                                                                                                 you should
                                                                                                 backup the
                                                                                                 server from
                                                                                                 a bootable
                                                                                                 CD. The
             Picture 23: Trivadis backup module, exemplary                                       script uses
                                                                                                 standard
UNIX utilities, except you want to use partimage as the backup solution.

Category                         Bare metal backups (Disaster)
Tools used                       partimage or tar
Gui                              none (command line perl script)
Requirements                     Perl, partimage, tar, gzip, bzip2
Schedule able                    yes, via crontab
Configuration                    Command line options




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 24 / 34
9.1 Features
            Automatically creates a copy of all partition tables of all local disks
            Auto select of all OS related partitions or Logical Volume Groups
            Backups can be done using partimage or simple tar archives
            partimage or compressed tar archive creation is possible
            Stores the disk, LVM and software raid configuration in separate files


9.2 Advantages
            Backups can be done onto a remote Server
            Only standard Linux utilities needed, except you use partimage
            Just one script to backup the server
            Backups can be done on running server


9.3 Disadvantages
            Script requirements must be resolved manually
            Restore is only manually possible
            No incremental or differential backups


9.4 Usage
The backup can be executed using one command. To display a short help, use -h or --help option:

BackupPartimage.pl -h:
Usage: BackupPartimage.pl [ -q ] [ --server <[login@]IP or hostname> ] [ --mbr </dev/mbr disk> ] [ --savepath <path for saves> ]
    [ --vg <LVM volume group> [ --vg ... ] ] [ --part <partition> [ --part ... ] ] [ --remotepath <path on remote server> ]
               [ --type <partimage|tar> ] [ --all ] [ --onlyconfig ] [ --defaults ] [ --ztype <bz2|gzip|none> ]
    -q        => quiet mode, no warnings will be printed.
    --savepath => Where to save backups, default: '/data/backup'?
     --all    => All partition will be saved, if argument was given.
    --onlyconfig => No tar or partimage backups will be done, only config of LVM, RAIDs and disks will be made.
    --defaults => shows the script defaults and exits.
    --ztype    => zip format, allowed is bz2, gzip or none for uncompressed backups.

     Note: If partimage binary, '/usr/sbin/partimage', was not found on system, script switches automatically to tar backups!
     In case of any problems, try without '-q' option!!




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 25 / 34
 10 rdiff-backup




            Picture 24: rdiff-backup, exemplary

Using rdiff-backup it is possible to make full and incremental backups, but the same data occurs once only in
the backup archive. This means that the space in archive equals the size of the original (previous) data added to
the newly changed data on the system.

Category                          Data backups
Tools used                        rsync, ssh
Gui                               none (command line python script)
Requirements                      Python, librsync (dependency during installation of package)
Schedule able                     yes, via crontab
Configuration                     Command line options




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 26 / 34
10.1 Features
     Incremental backups
     Backups (from or) to remote servers
     Doesn't waste space in the backup archives, the same data appears only once
     Command-line tool


10.2 Advantages
     Restore with the same script possible
     Saves space, because of creating hard links for equal files


10.3 Disadvantages
     Only command line tool available
     Uses quite a lot of CPU resources (rsync)
     For incremental backups it's necessary to create hard links, so only one file system is possible for the
     archives




  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 27 / 34
10.4 Usage
In order to make remote backups or restores, rdiff-backup must be installed on both servers.

The syntax for remote folders is always like follows:
 rdiff-backup user@host::/var user@host::/media/wmax1/rdiff/var

Note: Both, source and destination can be on remote servers. All commands are possible over a network
archive, if you use the proper syntax.

        Backup (full): This example shows the backup command to backup /var into /media/wmax1/rdiff/var
        folder.
         Backup command, rdiff-backup ...:
         rdiff-backup --exclude /var/log --exclude /var/tmp –-exclude-sockets \
         /var /media/wmax1/rdiff/var

        Note: If you want to see, what's included in the backup, use –v5 and –-print-statistics option.

        Backup (incremental):
        Backup command, rdiff-backup ...:
         rdiff-backup --exclude /var/log --exclude /var/tmp –-exclude-sockets \
         /var /media/wmax1/rdiff/var

        Note: The command for incremental backups is the same. The utility checks the destination first for an
        existing backup, before it starts an incremental backup.

        Show available backup (in time):
        Backup command, rdiff-backup -l ...:
         rdiff-backup -l /media/wmax1/rdiff/var
         Found 1 increments:
            increments.2008-03-28T14:16:04+01:00.dir Fri Mar 28 14:16:04 2008
         Current mirror: Mon Mar 31 10:31:38 2008




    info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 28 / 34
   Compare: This example shows the compare between the current system (/var folder) and the backup
   archive (/media/wmax1/rdiff/var).
    Backup command, rdiff-backup --compare ...:
    rdiff-backup --compare /var /media/wmax1/rdiff/var
    hanged: lib/dhcp/dhclient.leases
    changed: lib/init.d
    changed: lib/init.d/daemons

    ...

    new: log
    new: log/Xorg.0.log
    new: log/Xorg.0.log.old

    ...

    deleted: tmp/rpm-tmp.851
   This option is quite useful to check the changes between the (different) archives and the actual or given
   time, see –-compare-at-time 1D option. For further information, please consult the man page.

   Restore: There are different ways to restore a file or folder from archive. Using rdiff-backup, or just
   UNIX commands (like cp, scp, ...), if you need the newest backup to restore.
   Backup command, rdiff-backup --restore-as-of <time> \
   <archive file or folder> <restore location>:
    rdiff-backup --restore-as-of 1D \
    /media/wmax1/rdiff/var/log /var/log
   This would restore a backup from the last day into original destination. It's also possible to redirect a
   restore.

   Remove any old backup from archive:
   Backup command, rdiff-backup –-remove-older-than <time> <archive>:
   rdiff-backup --remove-older-than 2W /media/wmax1/rdiff/var
   This would remove any backup older than 2 weeks. The restore does not create any directory for you. It
   does not recursive restore directories, if you select files only!




info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 29 / 34
 11 rsnapshot




             Picture 25: rsnapshot, exemplary

The perl based rsnapshot is very similar to rdiff-backup, the main difference is the used scripting language.
Another important difference is that rsnapshot uses a configuration file to automate the backups through crontab.

Category                          Data backups
Tools used                        rsync, ssh
Gui                               none (command line perl script)
Requirements                      Perl, ssh, rsync, optionally logger, du, cp
Schedule able                     yes, via crontab
Configuration                     One file: /etc/rsnapshot.conf




      info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 30 / 34
11.1 Features
     Incremental backups
     Automatic remove of older backups, see interval option in configuration
     Fully crontab support (It creates a logfile, and works silently)
     Configuration is done in one file


11.2 Advantages
     Very good for automated backups
     Customizable logger
     Takes care using USB or external disks, see no_create_root option


11.3 Disadvantages
     Not a lot of interactivity
     Only in command line available
     Uses quite a lot of CPU resources (rsync)
     For incremental backups it's necessary to create hard links, so only one file system is possible for the
     archives




  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 31 / 34
11.4 Usage

     Configuration:
     Only, tho most important settings are shown next.
     Main and only configuration file of rsnapshot, /etc/rsnapshot.conf:
      ...

      ###########################
      # SNAPSHOT ROOT DIRECTORY #
      ###########################

      # All snapshots will be stored under this root directory.
      #
      snapshot_root        /media/wmax1/bt

      # If no_create_root is enabled, rsnapshot will not automatically create the
      # snapshot_root directory. This is particularly useful if you are backing
      # up to removable media, such as a FireWire or USB drive.
      #
      no_create_root        1

      ...

      #########################################
      #        BACKUP INTERVALS            #
      # Must be unique and in ascending order #
      # i.e. hourly, daily, weekly, etc. #
      #########################################

      interval    hourly    4
      interval    daily     7
      interval    weekly    4
      #interval   monthly   3

      ...

      ###############################
      ### BACKUP POINTS / SCRIPTS ###
      ###############################

      # LOCALHOST
      #backup /home/                  localhost/
      backup /etc/                    localhost/

      # Ubunto test laptop:
      backup root@172.16.39.89:/etc                        ubuntu



            snaphost_root:
            This is the backup archive place, all backups will written here.
            no_create_root:
            If the backup archive is on a mountable drive, you can enable this option. If set and the drive is not
            mounted, no creation of the root is forced, the script aborts.
            interval:
            In this example, four backups will be rotated, if hourly backup was called.
            Seven backups will be rotated, if daily backup was called.
            Four backup will be rotated, if weekly backup was called
            Make sure, you configure the crontab properly:
             Corresponding crontab:
            0 */6 * * * /usr/bin/rsnapshot hourly
            0 1 * * 1-6 /usr/bin/rsnapshot daily
            0 1 * * 0 /usr/bin/rsnapshot weekly

            backup:
            All backups are defined with this directive. If no further option was given in command line, all this
            backups are executed.


  info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 32 / 34
   You can test the configuration using rsnapsot configtest.

   Backup:
   Backup command, rsnapshot ...:
    # rsnapshot -v hourly
    echo 10406 > /var/run/rsnapshot.pid
    /usr/bin/rm -rf /media/wmax1/bt/hourly.1/
    /usr/bin/cp -al /media/wmax1/bt/hourly.0 /media/wmax1/bt/hourly.1
    /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded /etc \
       /media/wmax1/bt/hourly.0/localhost/
    /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded \
       --rsh=/usr/bin/ssh root@172.16.39.89:/etc \
       /media/wmax1/bt/hourly.0/ubuntu
    touch /media/wmax1/bt/hourly.0/
    rm -f /var/run/rsnapshot.pid


   Restore:
   No restore option is actually included in the rsnapshot script. In case of restore, you must do this
   manually! You can use scp, cp or rsync to restore any file or folder.

   Logging:
   If nothing else is configured, the logfile is in default location /var/log/rsnapshot.
    Logfile example, /var/log/rsnapshot:
    ...

    [01/Apr/2008:10:55:23] /usr/bin/rsnapshot -v hourly: started
    [01/Apr/2008:10:55:23] echo 10204 > /var/run/rsnapshot.pid
    [01/Apr/2008:10:55:23] /usr/bin/cp -al /media/wmax1/bt/hourly.0 /media/wmax1/bt/hourly.1
    [01/Apr/2008:10:55:28] /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded /etc
    /media/wmax1/bt/hourly.0/localhost/
    [01/Apr/2008:10:55:28] touch /media/wmax1/bt/hourly.0/
    [01/Apr/2008:10:55:28] rm -f /var/run/rsnapshot.pid
    [01/Apr/2008:10:55:28] /usr/bin/rsnapshot -v hourly: completed successfully
    [01/Apr/2008:11:05:40] /usr/bin/rsnapshot -v hourly: started
    [01/Apr/2008:11:05:40] echo 10406 > /var/run/rsnapshot.pid
    [01/Apr/2008:11:05:40] /usr/bin/rm -rf /media/wmax1/bt/hourly.1/
    [01/Apr/2008:11:05:46] /usr/bin/cp -al /media/wmax1/bt/hourly.0 /media/wmax1/bt/hourly.1
    [01/Apr/2008:11:05:51] /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded /etc
    /media/wmax1/bt/hourly.0/localhost/
    [01/Apr/2008:11:05:51] /usr/bin/rsync -ax --delete --numeric-ids --relative --delete-excluded --rsh=/usr/bin/ssh
    root@172.16.39.89:/etc /media/wmax1/bt/hourly.0/ubuntu
    [01/Apr/2008:11:06:11] touch /media/wmax1/bt/hourly.0/
    [01/Apr/2008:11:06:11] rm -f /var/run/rsnapshot.pid
    [01/Apr/2008:11:06:11] /usr/bin/rsnapshot -v hourly: completed successfully




info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 33 / 34
 12 Conclusion
Depending on what kind of information/data needs to be backup, Open Source softwares offer serious and
reliable alternatives to “closed-source” products. If you need enterprise ready backup software, Bacula appears
to be the most serious candidate. Trivadis did a few Bacula installations on customer sites with very good
feedback.

However if no “backup to tape” is required, BackupPC can help a lot. In this case the “backup server” strategy
will be followed. Trivadis uses BackupPC as a part of its backup strategy since three years and advices this
solution to save all UNIX based clients with less than 100GB data.

rsnapshot or rdiff-backup can also be considered if only a few servers need to be backed up. Because both of
these tools use rsync for synchronization, they are quite reliable and proved it, in the practice.

Also for disaster/recover purposes, Open Source provides strong backup recovery solutions. We recommend
SystemRescueCD, mondoarchive or partimage which allow to design strong and reliable backup concepts. If
you have to recover a server, SystemRescueCD can be very flexible and is used by Trivadis since many years.


                           OpenSource backup tools comparision
  450

  400

  350

  300
                                                                                                Reliability
                                                                                                Usability
  250                                                                                           Disaster backups
                                                                                                Data backups
  200                                                                                           Enterprise

  150

  100

   50

    0
    Bacula      BackupPC    mondorescue   rdiff-backup   rsnapshot   partimage   System-
                                                                                 RescueCD


Picture 1: OpenSource software comparision chart




    info@trivadis.com . www.trivadis.com . Info-Tel: 0800 87 482 347 . Date 24. Oct. 2008. Page 34 / 34

				
DOCUMENT INFO
Categories:
Tags:
Stats:
views:3
posted:3/28/2012
language:
pages:34