How can I automate the backup a PmWiki installation with the program
Updated: Tue Jul 10 22:33:06 CDT 2012 by tamouse
Description and Usage
Backups are a hugely important thing when you have a wiki full of information. PmWiki is not a difficult system to keep backed up, as everything is stored in flat files on the server. There have been a few solutions proposed. This method uses the a rolling backup method, using a hard-linking scheme to only pull over the changes in each backup iteration.
Alternately, you can make a clone from the git repository at gitorious and run
make in the directory.
Wikibackups.pl uses the following perl modules:
Obtain the first two from cpan.org and install as usual.
wikibackups.plto some place on your
PATH, such as
wikibackups.1to a useful man path (
- edit the configuration file in
$HOME/.wbu.rcby putting in the
TARGETparameters, and adjusting
exclusionsas you see fit. You can put information for multiple wikis in the same configuration file, just replicate the structure that is there and name each wiki appropriately.
- note that
$HOME/.wbu.rcis a YAML file. MAKE SURE YOU DON'T USE ANY TABS IN THIS FILE
- note that
- ensure the
TARGETdirectory exists; create it if it does not.
Running the script
wikibackups.pl [-c config] [-n] [-v] [-d] wikibackups.pl [-h] wikibackups.pl [-V]
- -c config - specify an alternate configuration file. The default is "$HOME/.wbu.rc".
- -n - dry run only. If used with -v will show the list of files which rsync would have transfered.
- -v - verbose mode. Will print out what is going on during the backup, including the output from the rsync command.
- -d - debug mode. Will print out all sorts of internal structure information. Useful for checking your configuration file. If you want to check your configuration, make sure to set -n and -v as well.
- -h - help. Prints a usage message.
- -V - version. Prints the script version.
I strongly encourage thorough testing of the backup mechanism before automating it. In addition, make sure you can recover files you want from the backups generated.
This recipe will create backup versions, but it won't delete any. You may want to have something that will prune off old backup versions. The entire backup set should not take that much space unless your wiki experiences a great deal of entropy, since using the hard linking feature of *nix only takes extra space with files actually change, but it can be disconcerting to see many directories.
See the man page, html page or txt page in the
wikibackups installation for more information on configuring and running the backup.
Updated: tamouse October 23, 2011, at 02:26 PM
rsync is an extremely useful solution to backing up files to or from a remote location, provided you have ssh access to the remote machine. The rsync man page lists tons of options and examples for doing this well.
A simple example of using rsync to replicate your wiki:
$ rsync -av user@server:path/to/wiki/ local/path/to/backup
This will bring over everything, including the pmwiki software and such, which you probably don't really need in a backup situation.
Inclusions and Exclusions: rsync filtering
Luckily, rsync let's you include and exclude things by using an
--exclude-from parameter that you can drop file glob patterns
It's a bit arcane, and the man page is less than helpful, but I've come up with a set of filter rules that seem to work for backing up the important content of your wiki.
Create a file called, say,
exclusions, and fill it with:
# Exclude stuff from pmwiki except some + /cookbook/ + /wiki.d/ - /wiki.d/.flock - /wiki.d/.pageindex - /wiki.d/.lastmod - /wiki.d/*,del-* - /wiki.d/*/*,del-* + /pub/ + /local/ + /uploads/ - /* - **~ - **,v - **.bak - **.tgz - **.zip - **.gz - **.Z
This will allow the files matching include patters (prefix "+") and prevent the files matching exclude patterns (prefix "-") from being included in the rsync. Note well that the include patterns need to come before the exclude patterns. Here's what's going on above:
|include the cookbook directory at the top level of the directory tree (i.e. if cookbook reappears in a subdirectory).|
|include the wiki.d directory, again at the top level only|
|exclude the .flock file under wiki.d|
|exclude the .pageindex|
|exclude deleted pages|
|exclude deleted pages in a Group direcotry setting|
|include the pub directory|
|include the local directory|
|include the uploads directory|
|exclude all other directories and files in root.|
This last line here is really the key to the whole thing. If you don't have this line, you'll still get most everything in the installation. It basically says "exclude everything else from here".
What follows are more miscelaneous matching patterns for various kinds of files that may appear in the included directories up above.
|typical editor backupt file, such as emacs. the double star means match everything.|
|RCS vault file|
|another type of backup file, usually from filters|
|tarballs, generally don't want to include those in your personal arvhive.|
|zip files, similar to tarballs|
|gzipped files, frequently tarballs are ended .tar.gz|
(Note: you may have good reason not to exclude some of the types of files I've shown above. Feel free to modify the above list; add and subtract entries as needed.)
Once you have your exclude file set up, then run the following command:
$ rsync -av --exclude-from=exclusions user@server:path/to/wiki local/path/to/backup
Rsync has a nice parameter,
--dry-run, which lets you see what
it's going to do before you commit to the download. This is great for
checking yout inclusions and exclusions to make sure nothing comes over
that you don't want.
There are many more options to rsync that can make it the ideal backup solution, including making incremental and full backups, etc.
For a more robust backup setup using rsync, see http://www.sanitarium.net/golug/rsync_backups_2010.htm as it outlines a rather nice rolling backup. -- tamouse July 10, 2012, at 10:59 AM
Just keep in mind the 3 most important things:
BACKUPS, BACKUPS, BACKUPS
Change log / Release notes
- Version 0.1: 2012-07-10
If the recipe has multiple releases, then release notes can be placed here. Note that it's often easier for people to work with "release dates" instead of "version numbers".
See discussion at BackupWithRsync-Talk