Update 1 [2016/09/23]: OmniBackup now officially supports GNU/Linux. More info…
Update 2 [2016/09/23]: Official documentation moved to GitHub which means this guide won’t be maintained anymore and maybe out of date or inaccurate.
A week ago was System Administrator Appreciation Day. It is celebrated on the last Friday in July and it has been celebrated since July 28, 2000. But, system administrators know not all days are like that day. They face many hard times and struggles during their careers and the worse of them all is either a security breech or data loss.
For so many years I’ve been writing and maintaining backup scripts on and on, for every single database I added, for every single directory with critical data, or any other service I was running on every new server I got my hands on. In the end, I found myself ended up in a pile of backup scripts and multitudinous cron entries which was a nightmare to keep track of. I even had to manage the schedule so that two backup scripts do not run at the same time. Even more, I had to manually track the backups to see whether they were successful or not. Also, someone has to manually delete the old ones to make rooms for the next ones.
Therefore, I came up with an elegant solution to replace the old process which I found exceptionally error-prone. An end to all my hardships which I call OmniBackup. At last, I’m able to confidently keep abreast of all the ever-growing data that I have to keep safe.
“So, what exactly is OmniBackup?” you may ask. “A fair question,” I would say. OmniBackup is a MIT licensed Bash script which delivers the following set of features:
- Configuration and customization of backup mechanism through JSON
- Support for OpenLDAP backups
- Support for PostgreSQL backups as a whole or per database
- Support for MariaDB and MySQL backups as a whole or per database
- Support for filesystem backups with optional ability to follow symbolic links
- Support for pluggable customized scripts to extend OmniBackup functionality beyond its original design which allows support for many different backup scenarios that has not been built into OmniBackup, yet
- Backup file name tagging which allows including date or host name in the archive name
- Online backup without a prerequisite to suspend any service
- Support for customized backup tasks priority order
- Support for multiple backup servers
- Ability to always keep a copy of backups offline
- Ability to keep a copy of backups offline if no backup server is available, or in case of an error such as a file transfer error
- Secure file transfer through SSH / SCP protocol
- LZMA2, gzip and bzip2 compression algorithms along with different compression levels to maintain a balance between speed and file size
- Ability to preserve permissions inside backup files
- Support for symmetric cryptography algorithms AES-128, AES-192 and AES-256 (a.k.a Rijndael or Advanced Encryption Standard)
- Random passphrase generation for encrypted backups with variable length and patterns or a unique passphrase for all backups
- Support for RSA signatures to verify the backup origin and integrity
- Passphrase encryption using RSA public keys for individual backup servers
- Backup integrity verification by offering hash algorithms such as MD4, MD5, MDC-2, RIPEMD160, SHA, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512 and WHIRLPOOL
- Optional Base64 encoding
- System logs and a standalone log file including all details
- Reporting through email to a list of recipients with ability to include passphrases
- Customized mail subject for successful and failed backup operations
- Customized support message for reports
- Crontab integration
- Custom temporary / working directory
- Automatic and smart clean-up ability
- One instance only policy which avoids running multiple instances by mistake at the same time, therefore avoids system slow-down
- An example configuration file in JSON format to get you up and running
There is also a list of planned features and TODOs which did not make it into 0.1.0 release:
- Restore script
- GnuPG integration
- SFTP and FTP support
- Refactoring and code clean-up
- Any potential bug fixes
Disclaimer: Please be wary of the fact that this script has approximately 3.5 K lines of Bash code and devoured hell of a time from me to write and debug. You should also consider that this is my first heavy Bash experiment and I may not write quality code in the language since I’m a newcomer to Bash. I do not claim that OmniBackup is production ready, that’s why I did version the first release at 0.1.0. Also keep in mind that OmniBackup heavily relies on 3rd-party software which increases the chance for fatal bugs, therefore losing data. So, I provide OmniBackup without any warranties, guarantees or conditions, of any kind and I accept no liability or responsibility for any misuse or damage. Please use it at your own risk and remember you are solely responsible for any resulting damage or data loss.
Credits: Many thanks go to my fellow and long-time friend, Morteza Sabetraftar for his help and ideas without whom OmniBackup lacked features or quality. Another kudos goes to my brother Amir by releasing me from shopping, cooking and house-cleaning without even mentioning it, an invaluable and priceless assistance that encouraged me even more to use my best endeavours to get this task done.
Please, feel free to clone and modify it as you wish. Pull requests for new features, improvements or bug fixes are also very welcome.
The rest of this post serves as a comprehensive guide on how to setup OmniBackup in order to backup and restore all your critical data in a production environment.
[Read More...]