Changes

From Amahi Wiki
Jump to: navigation, search
9,958 bytes added ,  03:23, 23 June 2020
{{NeedsUpdate}}
[http://rsnapshot.org/ '''''rsnapshot'''''] is an excellent backup application and has been around for a long time and is extremely stable.
 
Why are backups important? Well, hardware failures, accidental file deletion(s), files become corrupted, etc.. Backups can help recover data where applications are being updated/upgraded. Enough reasons?
 
Where should backups be stored? Due to the inevitable failure of hardware it is desirable to store backups on a separate system from where the production data is stored. Or possibly to an external medium such as an external USB disk drive. This article discusses backing up to an external USB drive.
'''Please Note:'''
''rnapshot is an excellent backup application and has been around for a long time and is extremely stable. This article discusses doing backups to an external USB drive that is formatted for the "ext3" filesystem type. It '' depends on the attributes that are unique to Linux/Unix (i.e. ext3, ext4) filesystems only. If backups to "CIFS " (Samba a.k.a. smb, fat, fat32, vfat, ntfs) filesystems are required a different backup application '''must''' be used (i.e. [http://rdiff-backup.nongnu.org/ '''''rdiff-backup''''']).  {| border="0"|- valign="top"| border="0" | [[File:Bk_backup-recovery.png‎ |left]]||| border="0" | Here is a good book that discusses backups, recoveries, rsnapshot and rdiff-backup:<br/><br/>"'''''Backup & Recovery: Inexpensive Backup Solutions for Open Systems''''', by W. Curtis Preston".|}
__TOC__
=Another == Preparing an External USB Drive === Off-the-shelf external USB drives normally are setup to work with computers that utilize CIFS type filesystems. rsnapshot uses a "hard links" feature that is unique to Linux/Unix filesystems (ext3). "Hard Links" are not supported on filesystems such as: '''"smbfs"''', '''"Samba (SMB)"''', '''"FAT"''', '''"FAT32"''', '''"NTFS"''', '''"CIFS"''', '''"vfat"'''. Please note that "Amahi Greyhole" mounts Amahi shares that will look like "local disk drives" '''''but''''' these are Samba (SMB) mounted shares. An external disk drive will have to be prepared for a Linux/Unix filesystem, in particular ext3. Why? ext3 has been around for sometime, is stable, and "Windows" computers have Open Source drivers available that can be installed to gain access to an ext3 drive. Prepare an external drive for ext3 as follows: * The external USB drive used in this example is a 1 Terabyte (1 TB) Seagate drive.* Plug the drive in to a USB connector and power it up.* Using a "terminal" session, login in using the "root" user ID.* The "device name" of the drive needs to be determined.  Enter the following command: dmesg The following information is displayed:<pre>[11122.304178] usb 2-4: new high-speed USB device number 5 using ehci_hcd[11122.422152] usb 2-4: New USB device found, idVendor=0bc2, idProduct=3320[11122.422165] usb 2-4: New USB device strings: Mfr=2, Product=3, SerialNumber=1[11122.422173] usb 2-4: Product: Expansion Desk[11122.422180] usb 2-4: Manufacturer: Seagate . . . . .[11123.516813] sdb: sdb1</pre>* The "device name" is "sdb1".* The USB drive needs to be "unmounted".  umount /dev/sdb1 * Format drive for ext3 filesystem and set drive label name as "usbdisk".  mkfs -t ext3 -v -L usbdisk /dev/sdb1  :'''Please Note: THIS WILL ERASE ALL DATA ON /dev/sdb1, MAKE SURE THIS IS THE CORRECT USB DRIVE AND NOT ANOTHER DRIVE.''' The following information is displayed:<pre>mke2fs 1.42.5 (29-Jul-2012)fs_types for mke2fs.conf resolution: 'ext3'Filesystem label=usbdiskOS type: Linux . . . . .Writing superblocks and filesystem accounting information: done</pre>* Whenever a drive is formatted in Linux, about 5% is reserved of rsnapshot configuration the total space on amahi the drive for the operating system to continue using the hard drive to operate, even if it gets full. This is totally unnecessary for a USB external hard drive if it stores only data and not to run an operating system.  Enter the following to remove the reserved space: tune2fs -m 0 /dev/sdb1 The following information is displayed: tune2fs 1.42.5(29-Jul-2012) Setting reserved blocks percentage to 0% (0 blocks) * The external drive is now ready for use. === Mounting an External USB Drive === There are two methods of mounting an external USB drive. The first, mounts the drive automatically each time the server "boots" or "reboots". The second, is mounted manually by the "root" user via the "command-line". Before proceeding a couple of information items need to be gathered: * In the previous section the device name found found via the "dmesg" command. The device name found was "sdb1".* We set the set the external drive's "label". It was set to "usbdisk".  To find the label at anytime via the "root" user ID enter: e2label /dev/sdb1 The following information is displayed: usbdisk Both methods of mounting will be controlled by the server's "fstab" configuration file. The fstab (a.1k.a. file systems table) file is a system configuration file containing lists of available disks and disk partitions, and indicates how they are to be initialized or otherwise integrated into a system's file system. When the "mount" command is used, the fstab file is read to determine which options should be used when mounting a specified device.  ==== Auto-Mounting ==== Auto-mount of an external USB drive will occur when a server "boots" or "reboots". Provided the fstab configuration file is properly configured. A powered up USB drive that is plugged in after a server has completed the boot process is not auto-mounted. However, using a "terminal" session with the "root" user ID, and issuing a "mount -a" command will mount an external USB drive. Configure fstab to auto-mount an external USB drive do the following: * Start a "terminal" session login to the "root" user ID. * Navigate to the "etc" directory with the "cd" command and make a backup of the current fstab file. cd /etc cp /fstab /fstab_bak * Using a preferred file editor (i.e. vi, nano), open the fstab file. fstab * Near the bottom of the and after the other fstab entries, add the following:  LABEL=usbdisk /media/usbdisk ext3 defaults 0 0 * Now save the fstab file and exit the editor.* Navigate to the /media directory and create a "usbdisk" directory. This is where the external drive will be mounted, this is called a "mount point". cd /media mkdir usbdisk
In this example automated backups are made to an * Make sure the external usb driveUSB cable is connected to the server and is powered up.Plug in * Reboot the server.* Remember the disk label set earlier when formatting the USB drive? During the boot process when the "LABEL" is read, this instructs the system to find a hard disk with the label "usbdrive", and then mounts it will probably to the mount automatically under point, "/media/yourdisklabelusbdisk".
First login as root ==== Manual Mounting ====Mount of an external USB drive will not occur when a server "boots" or "reboots" when the fstab configuration file is configured for manual mounting. A powered up USB drive that is plugged in after a terminal (putty or ssh root@hda)and typeserver has completed the boot process is also not mounted.
Configure fstab for a manual mounting of an external USB drive with the following:mount
in * Start a "terminal" session login as the output, find which device is your usbdrive, possibly /dev/sdb1"root" user ID.
=Format usb disk=You need * Navigate to format the disk for rsnapshot to be able to use hard links. An external drive mounted via '''"smbfsetc"''', '''directory with the "FAT"''', '''"NTFS"''', or '''"cifs"''' does not allow "hard linkscd"command and make a backup of the current fstab file. It is not possible to create "snapshot-like" backups on FAT or NTFS formatted volumes, nor SMB cd /etc cp /fstab /CIFS as these filesystems do not support hard-linking.fstab_bak
:umount /media/yourdisklabel * Using a preferred file editor (i.e. vi, nano), open the fstab file. nano fstab
* Near the bottom of the and after the other fstab entries, add the following:mkfs -t ext3 -v -L LABEL=usbdisk /devmedia/sdb1usbdrive ext3 rw,noauto,users,exec,noatime 0 0
THIS WILL ERASE ALL DATA ON * Now save the fstab file and exit the editor.* Navigate to the /devmedia directory and create a "usbdisk" directory. This is where the external drive will be mounted, this is called a "mount point". cd /sdb1, media MAKE SURE THIS IS YOUR mkdir usbdrive* Make sure the external drive USB DRIVE AND NOT ANOTHER DRIVEcable is connected to the server and is powered up.* Reboot the server.* Start a "terminal" session login as the "root" user ID.* To manually mount the external drive issue the following command: mount -L usbdisk* The system reads the fstab, when it finds a line entry with the label "usbdrive", it mounts the drive the mount point, "/media/usbdisk".
=Install rsnapshot=:yum install rsnapshot== Unmounting ====
With both mounting methods the external drive can be unmounted by the following:mkdir -p /media/usbdisk/.private/.snapshots
* Start a "terminal" session login as the "root" user ID.* Issue the following command to unmount the external drive:chmod 0700 umount /media/usbdisk/usbdrive* Should an error occur indicating that the drive is busy and cannot be unmounted enter the command below. Then try the unmount command again.private fuser -km / media
:chmod 0755 /media/usbdisk/===Installing rsnapshot===rsnapshot is available as a package in both the Fedora and Ubuntu Linux distributions.private/The installation shown here is based on Fedora.snapshots/
=Mount your usbdisk permanently=To Mount your usbdisk permanently (so the backup will work after * Open a reboot) do "terminal" session and login using the following"root" user ID:
:cp /etc/fstab /etc/fstab_bak yum install rsnapshot
* Now create the directory structures for storing the backups:nano /etc/fstab
Add to this file mkdir -p /media/usbdisk/.private/.snapshots chmod 0700 /media/usbdisk/.private/ chmod 0755 /media/usbdisk/.private/.snapshots/
:LABEL=usbdisk /media/usbdisk ext3 defaults 0 0== Configuring rsnapshot ===This section shows how to configure the rsnapshot.conf configuration file.
Save * Open a "terminal" session and login to the file"root" user ID.
=Quick rsnapshot Recipe=* Move to the "/etc" directory.
Here is a quick recipe to configure rsnapshot. cd /etc
These are * Make a backup copy of the critical settings rsnapshot configuration file. Why? So that there is a reference just in /etc/rsnapshotcase.conf:
snapshot_root /mnt/1tb-external/snapshots/ interval daily 14 interval weekly 4 interval monthly 4cp rsnapshot.conf rsnapshot.conf.bkup
This will create * Now using a series of directories called dailyfavourite editor, open the "rnapshot.0 through dailyconf" file.13weekly<br/>'''Please Note:''' Items in the configuration file are separated by "tab" characters and NOT multiple spaces.0 to weekly.3 ... etc nano rsnapshot.conf
=Modify rsnapshot configuration=:nano /etc/rsnapshot* Find the "SNAPSHOT ROOT DIRECTORY" section and change "snapshot_root" to path where the backups will be stored.conf
make the following modifications to the file rsnapshot snapshot_root /media/usbdisk/.conffor blank spaces use TABS!private/.snapshots/}}
:snapshot_root /media/usbdisk/* Find the "no_create_root" section and remove the "#".private/This will allow rsnapshot to create the "snapshot root" directory.snapshots/
: no_create_root 1
:* Find the "BACKUP INTERVALS" section and change the line items. What does this all mean? The "#" character at the beginning of each line means the configuration line item will not be enabled. Remove the "#" character as required. For example, when enabled the "interval hourly 6daily" line will be active and 7 days of backups will be kept.
: interval hourly 6 interval daily 7 interval weekly 4 interval monthly 12
* Find the "GLOBAL OPTIONS" section. This section defines how much information will be saved in the rsnapshot "log" file. The options are:interval weekly Verbose level, 1 through 5. 1 Quiet Print fatal errors only 2 Default Print errors and warnings only 3 Verbose Show equivalent shell commands being executed 4 Extra Verbose Show extra verbose information 5 Debug mode EverythingChange the following options as required: verbose 3 loglevel 3
:interval monthly 12* Find the "logfile" line and change the line item. After rsnapshot has completed backups this is where the resulting "log" file is stored. Change this line as required.
:verbose 3 logfile /var/log/rsnapshot}}
:backup * Find the "BACKUP POINTS /varSCRIPTS" section. Add changes as seen below. This is the section that defines the "key directories" that will be backed up. Please note the "localhost/hda", this represents the server that it being backed up. "localhost/files/docs/ localhost" is a default name given to the current server. Change this name as required, say "amahiBox/"?
: backup /var/hda/files/docs/ localhost/ backup /var/hda/files/pictures/ localhost/ backup /var/hda/files/movies/ localhost/ backup /var/hda/files/music/ localhost/ backup /var/hda/files/other_shares_you_made localhost/
:backup /var/hda/files/movies/ localhost/* Save the "rsnapshot.conf" file.
:backup /var/hda/files/music/ localhost/=== Test the Configuration Changes ===* Now the changes to the rsnapshot configuration file have to be tested. rsnapshot has a feature to test changes to make sure things have been setup correctly.
:backup /var/hda/files/othersharesyoumade localhost/* Enter the following on the command-line to initiate the test.
Save the file rsnapshot configtest}}
* When the test completes the following should be displayed:rsnapshot configtest
this should return: syntax Syntax OK
=Email reporting=* If an error occurs the rsnapshot configuration file will need to be checked.
for email reporting on your backups:===Automate Backups ===Backups can be automated by creating a "cron job" that can be executed by the system at predetermined intervals.
:cp /usr/share/doc/rsnapshot*/utils/rsnapreportUsing a "terminal" session and "root" user ID a "cron job" configuration can be created to perform automated backups.pl /rootCreate a "cronjobs" text file as follows: crontab -l > cronjobs
:chmod 744 rsnapreport* Using a favourite text editor open the cronjobs file.pl
=Automate backup with cronjob= nano cronjobs
* Add to the cronjobs file for rsnapshot:crontab -l > cronjobs
:nano cronjobs
add to this file for running 0 */4 * * * /usr/bin/rsnapshot hourly 30 23 * * * /usr/bin/rsnapshot daily 00 23 * * 1 /usr/bin/rsnapshot and a weekly email report on 30 22 1 * * /usr/bin/rsnapshot:monthly
:0 */4 * Save the cronjobs file.* * /usr/bin/rsnapshot hourly 2>&1 | /Now update the "root/rsnapreport.pl > /root/rsnapreport" user IDs "crontab" with the following:
:30 23 * * * /usr/bin/rsnapshot daily crontab cronjobs}}
:00 23 * * 1 /usr/bin/Reboot the server and rsnapshot weeklywill be operational.
===Email reporting===* To email a report of backups:30 22 1 * * /usr/bin/rsnapshot monthly
:00 22 * * 6 cp /usr/binshare/doc/rsnapshot du >> */rootutils/rsnapreport | nail –r "somereturnadress@provider.com" -s"HDA backup report" -S smtp=smtp.yourprovider.com youremail@provider.com < pl /root/ chmod 744 rsnapreport.pl'''*''' above indicates an rsnapshot version number
Save * Add the filefollowing to crontab for a weekly email report on rsnapshot:
<pre style="white-space:crontab cronjobspre-wrap; word-break: keep-all;">0 */4 * * * /usr/bin/rsnapshot hourly 2>&1 | /root/rsnapreport.pl > /root/rsnapreport00 22 * * 6 /usr/bin/rsnapshot du >> /root/rsnapreport | nail –r "somereturnadress@provider.com" -s"HDA backup report" -S smtp=smtp.yourprovider.com youremail@provider.com < /root/rsnapreport</pre>
Rsnapshot should now be operational'''Please Note:''' The above information maybe out of date due to technology changes. Please feel to update as required.
===Make your backups available on clients (READ ONLY)===
If you want to make the backups accesible from your clients:
Create a //hda/backup share in the HDA webinterface
: chkconfig nfs --level 2345 on
add a read only NFS export:
: nano /etc/exports
add
: /media/usbdisk/.private/.snapshots/ 127.0.0.1(ro,no_root_squash)
Save file
Unfortunately mounting an NFS share in fstab did not work on my machine after a reboot, so I chose an alternative configuration that mounts the share later in the booting process:
: nano /etc/rc.local
Add
: mount -r -t nfs localhost:/media/usbdisk/.private/.snapshots/ /var/hda/files/backup/
save file
[[Category: Apps]]
12,424

edits