Backup and Recovery of ownCloud

Have you been backing up your current ownCloud instance? If not, perhaps it's time? Think of all that time spent doing the setup and all that important data now stored in your ownCloud. Do you really want to take a chance losing the ownCloud configuration and all that stored data? Don't make it a "round to it" make it a "do it".

Following information will be provided to "backup and recover" data specific to ownCloud. No attempts will be made to discuss "philosophies" for backup and recover. The rsnapshot application will be used to backup data for an ownCloud instance and stored files. rnapshot does not perform the recovery of data and files. Recovery from the data backups will be done by other methods. Additionally, information provided is based on a Fedora platform.

The "web-apps" (red folder) directory structure is where application binaries for Amahi are stored. Below is the "owncloud-html branch". Under this branch is a collection of directories containing binaries for the ownCloud application. The "config" directory in particular is important as it contains files containing configuration and sharing information for an ownCloud instance. This directory is important when re-installing (recovery), updating, or upgrading (see "Upgrading ownCloud to New Version Releases") ownCloud.

ownCloud Data Directories

The "files" directory (red folder) referenced here is a typical Amahi shares structure. The "owncloud-data branch" is where physical files are stored in ownCloud. It is also important to note that "meta-data" for physicial files is kept in the ownCloud database (file sharing, groups). Therefore, it is important no only to backup the physical files but also the ownCloud database.

rsnapshot Backup Directories

The "backups" directory (red folder) shown here is located in the same filesystem location as the Amahi hda structures. This is not a recommended practice as the backups directory could be on the same hard disk(s) as the hda directories, it is only intended as an example for this article only. Normally the backups directory structure should be stored in a separate hard disk assembly (i.e. external drive) from the source hda structures.

The "daily.0" is created by the rsnapshot application to store "snapshot-like" backups. This particular directory name is typical of the doing a "daily" backup with rsnapshot. The directories "daily.x" and "daily.y" are just pseudo names where each value of x and y are normally numbers indicating later version numbers of rnsapshot backup sets. If rsnapshot daily backups are set for a 7 day retention the numbering would be 0 to 6, where daily.0 is the most recent backup set and 6 the latest set.

Below each "daily.?" backup set a directory (green folder) is created, normally intended for identifying a server name (default is localhost), for the purpose of this example "owncloud" is treated as the "server". Under the "owncloud server directory" structure there are branches containing backups of the ownCloud binaries, physical files, and "database dump". The "database_dump" directory contains a backup of the ownCloud database. rsnapshot can be configured to add additional branches under the owncloud server directory to store other backup data.

Configure

This section shows how to configure rsnapshot for backing up ownCloud binaries, physical files, and the ownCloud database information based on the previous key directories discussion. Minimal information for each configuration setting will be provided. For details refer to the Amahi rsnapshot wiki article or the rnapshot web site.

Follow the following configuration steps:

Make sure that rsnapshot is installed. Fedora and Ubuntu distributions have packages available for the software.
Open a "terminal" session and login to the "root" user ID.
Move to the "/var/hda" directory.
Create the "backups" directory and set permissions/ownership.

cd /var/hda
mkdir backups
chmod 755 backups
chown root:root backups

Move to the "/etc" directory.

cd /etc

Make a backup copy of the rsnapshot configuration file. Why? So that there is a reference just in case.

cp rsnapshot.conf rsnapshot.conf.bkup

Now using a favourite editor, open the "rnapshot.conf" file.

Please Note: Items in the configuration file are separated by "tab" characters and NOT multiple spaces.

Find the "SNAPSHOT ROOT DIRECTORY" section and change "snapshot_root" to path where the backups will be stored.

snapshot_root	/var/hda/backups/

Find the "BACKUP INTERVALS" section and change the line items. What does this all mean? The "#" character at the beginning of each line means the configuration line item will not be enabled. The "#" is know to set a line to be a "comment" line. The "interval daily" line will be active and 7 days of backups will be kept.

#interval	hourly	6
interval	daily	7
#interval	weekly	4
#interval	monthly	1

Find the "logfile" line and change the line item. The rsnapshot log file will be kept under the "backups" directory.

logfile	/var/hda/backups/logs/rsnapshot

Find the "BACKUP POINTS / SCRIPTS" section. Add changes as seen below. This is the section that defines the "key directories" source that will be backed up. See the "owncloud/" at the end of each line? Remember the previous "green" server folder? This is where the ownCloud binaries and physical files will be stored.

backup	/var/hda/files/owncloud owncloud/
backup	/var/hda/web-apps/owncloud/ owncloud/

Next find the line "Backup OwnCloud MySQL database" change as shown below. This set the location where the ownCloud "database dump" backup will be kept. Also the line shows the execution path for the database backup program script file that performs the database backup.

backup_script	/usr/local/bin/backup_mysql.sh	owncloud/database_dump

Save the "rsnapshot.conf" file and return to the command-line prompt.

Next database backup program script file has to be set up.

Copy database backup script to the "/usr/local/bin/".

cp /usr/share/doc/rsnapshot*/utils/backup_mysql.sh

* - indicates rsnapshot version number

Go to the "/usr/local/bin/" directory and set the script file permissions and ownership.

cd /usr/local/bin/
chown root:root /usr/local/bin/backup_mysql.sh
chmod 754 /usr/local/bin/backup_mysql.sh

Edit the "backup_mysql.sh" file.

Find the line "backup the database" and add the line has shown below. This will be the command that will be issued to do the database backup. The database dump file will be called "owncloud_db.sql".

/usr/bin/mysqldump -uowncloud -powncloud owncloud  > owncloud_db.sql

Find the line "# make the backup readable only by root" and add the line shown below.

/bin/chmod 600 owncloud_db.sql

Save the "backup_mysql.sh" file and return to the command-line prompt.

There rsnapshot is configured to backup ownCloud.

NOTE: If using ownCloud 6, change all instances of owncloud to owncloud6.

Test

Now the changes to the rsnapshot configuration file have to be tested. rsnapshot has a feature to test changes to make sure things have been setup correctly.

Enter the following on the command-line to initiate the test.

rsnapshot configtest

When the test completes the following should be displayed:

Syntax OK

If an error occurs the rsnapshot configuration file will need to be checked.

Backup

The following command-line entry will perform the backup:

rsnapshot daily

The first time that rsnapshot runs could take sometime, each subsequent run should be quicker (depending on data changes).
The following directories are created over time under the "backups" directory:

drwxr-xr-x   3 root root  4096 Feb 15 01:30 daily.0
drwxr-xr-x   3 root root  4096 Feb 14 01:30 daily.1
drwxr-xr-x   3 root root  4096 Feb 13 01:30 daily.2
drwxr-xr-x   3 root root  4096 Feb 12 01:30 daily.3
drwxr-xr-x   3 root root  4096 Feb 11 01:30 daily.4
drwxr-xr-x   3 root root  4096 Feb 10 01:30 daily.5
drwxr-xr-x   3 root root  4096 Feb  9 01:30 daily.6
drwxr-xr-x   2 root root  4096 Jan 20 09:38 logs

As mentioned before, notice that the "daily.0" directory is the most recent backup version.

Recovery

This section will discuss restoring, updating, and upgrading parts of ownCloud from backups. ownCloud recovery from backups use the "rsync" command and must be used by the "root" user ID on an Amahi server. "rsync" is run from the "command-line".

"rsync" has the following syntax on a command-line:

rsync [options] source-path destination-path

where "[options]" will be set to "-avr --delete" for ownCloud restores:

-a - archive, mode which includes the use of multiple rsync options.
-v - verbose, reports all activity rsync performs during a restore.
-r - recursive, include the sub-directories and their contents during a restore.
-- delete - delete extraneous directories and files from the destination path directories during a restore.

Restore ownCloud

The following steps will restore the ownCloud binaries and configuration.

Go to the Amahi Dashboard, Apps => Installed make sure that ownCloud is uninstalled.
If ownCloud is still installed, uninstall the current ownCloud instance. This will remove the ownCloud binaries and ownCloud database.

Go to the Amahi Dashboard, Apps => Available, install ownCloud.

Go to a "terminal" session and login to the "root" user ID.

Restore the ownCloud "config.php" file from the most recent backups with the following:

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/web-apps/owncloud/html/config/config.php /var/hda/web-apps/owncloud/html/config/

Restore the ownCloud "apps" directory from the most recent backups with the following:

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/web-apps/owncloud/html/apps /var/hda/web-apps/owncloud/html/

Restore the ownCloud database from the most recent backups using the database dump file:

1. Recreate the database (THIS IS EXTREMELY IMPORTANT):

mysqladmin -uowncloud -powncloud -f drop owncloud;
mysqladmin -uowncloud -powncloud -f create owncloud;

2. Restore the database:

mysql -u owncloud -powncloud owncloud < /var/hda/backups/daily.0/owncloud/database_dump/owncloud_db.sql

Login to ownCloud using a normal user ID and test ownCloud functionality (i.e. sharing) and applications.
If the prior testing was successful have other users test their ownCloud IDs.
The restore of ownCloud is now completed.

NOTE: If using ownCloud 6, change all instances of owncloud to owncloud6.

Restore ownCloud Files

The section describes the "rsync" command to be used to restore the ownCloud "files" directory structure.

Before doing the "files" restore make sure that there is enough disk storage room.
Enter the following "rsync" command to restore the "files" directory from the most recent backups. Depending on the amount of data stored the restore could take sometime.

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/files /var/hda/

NOTE: If using ownCloud 6 or greater, change all instances of owncloud to owncloud<major release number> (i.e. owncloud6).

ownCloud Update and Upgrade

This section briefly describes the "rsync" restore commands that can be used when doing an "upgdate" or "upgrade" of ownCloud.

	ownCloud Tip!
	Before going further please become acquainted with the Upgrading ownCloud to New Version Releases Wiki article.

Update

Restore the ownCloud "config.php" file from the most recent backups with the following:

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/web-apps/owncloud/html/config/config.php /var/hda/web-apps/owncloud/html/config/

Restore the ownCloud "apps" directory from the most recent backups with the following:

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/web-apps/owncloud/html/apps /var/hda/web-apps/owncloud/html/

The following command-line entry will restore the ownCloud database from the most recent backups:

mysql -u owncloud -powncloud owncloud < /var/hda/backups/daily.0/owncloud/database_dump/owncloud_db.sql

NOTE: For all releases on or after ownCloud 6, change all instances of owncloud to owncloud6.

Upgrade

Restore the ownCloud "config.php" file from the most recent backups with the following:

rsync -avr --delete /var/hda/backups/daily.0/owncloud/var/hda/web-apps/owncloud/html/config/config.php /var/hda/web-apps/owncloud/html/config/

The following command-line entry will restore the ownCloud database from the most recent backups:

mysql -u owncloud -powncloud owncloud < /var/hda/backups/daily.0/owncloud/database_dump/owncloud_db.sql

NOTE: For all releases on or after ownCloud 6, change all instances of owncloud to owncloud<major release number> (i.e. owncloud6).

Automate

Backup and recovery can be automated in order to reduce the amount of manual command-line work required for these operations.

A "root" user "cron job" could be created to perform backups at say "1:30 AM" everyday.

root user "crontab" configuration:

30 01 * * * /usr/bin/rsnapshot daily

Some system administrators will create a "nighttime backup script" cron job that is executed at say "1:30 AM" before a server is shutdown each night.

root user "crontab" configuration:

30 01 * * * /root/scripts/nighttime.bash

root user "nighttime.bash" script:

nighttime.bash                                                                                                    
#!/bin/bash
# Night time operations: Backup and Shutdown

# Backup Data
echo "Backup Start: "$(date)
/usr/bin/rsnapshot daily
echo "Backup Finished: "$(date)

# Send shutdown message
wall -n "$(date +%A) @ $(date +%T) system shutting down at scheduled time."
# Shutdown System
/sbin/shutdown -h now

Amahi ownCloud Backup/Recovery Tools

In order to make life easier for doing ownCloud backup and recovery a couple of tools might be of some benefit? The tools provided are based on the wiki articles "Backup and Recovery of ownCloud" and "Upgrading ownCloud to New Version Releases". The tools can be modified to meet the needs of a current ownCloud instance. The tools can be found at directory location, "/var/hda/web-apps/owncloud/html/scripts/". Please remember if the tools are "modified", whether they are left in place or put elsewhere, to back them up as in a next install of ownCloud they will be replaced with the Amahi package defaults.

Backup Tool

The backup tool is called,"oc-backup.sh". Currently it is configured to only do an "rsnapshot daily". The tool is run using the "root" user ID in a "terminal session".

Help information is available by:

/var/hda/web-apps/owncloud/html/scripts/oc-backup.sh -h

Resulting in the following:

Usage:
oc-backup.sh [option]

Options:
 -h         help
 --version  displays version of this script
 -d         do daily backup

The tool only allows one command-line option to be entered. Please note the "-d" option? This performs the daily backup process.

Recovery Tool

The backup tool is called, "oc-recover.sh". The recovery tool performs several different types of ownCloud recovery operations.

Help information is available by:

/var/hda/web-apps/owncloud/html/scripts/oc-recover.sh -h

Resulting in the following:

Usage:
oc-recover.sh [option]

Options:
 -h         help
 -f         restore data files
 -r         restore ownCloud
 -p         update ownCloud
 -g         upgrade ownCloud
 -rf or -fr restore ownCloud and ownCloud data files
 -pf or -fp update ownCloud and ownCloud data files
 -gf or -fg upgrade ownCloud and ownCloud data files
 --version  displays version of this script

The tool only allows one command-line option to be entered.

Main ownCloud Page

Amahi Wiki

Links

Personal tools

Navigation

Tools