smartly purge your old backup files on linux

if you backup your *nix box, eventually you'll get into the business of purging your old backup files to preserve disk space. a reasonable way to do this is to use the find command to identify old backups and delete them. you should, however, consider doing something a little smarter than this.

the basic find method is usually accomplished with a statement like the one below, run from cron.

$ find /var/myBackupDir -type f -mtime +20 -exec rm -v {} \;

this would simply find and delete all files in the directory /var/myBackupDir that are older than 20 days.

you may, however, want to do something a little more sophisticated. what if your database got seriously corrupted 21 days ago? using the above technique, you would have 20 days of corrupted files.

i wrote a utility for smart purging of old backup files, purgeBackup. this utility allows you to list your desired set of backup ages e.g. {1 day, 2 day, 3 day, 10 day, 20 day, 40 day, 100 day}, and keep only the files that most closely match this specification. the statement that you'd use for this would be:

$ purgeFiles --ages=1,2,3,10,20,40,100 --directory=/var/myBackupDir

you'd similarly run this from cron, and each day, allow it to asses the optimum set of backups to keep.

purgeBackup will run in a simulation mode (no deletion) until you specify the --force argument, regardless, please exercise caution when using this utility.

here's the purgeBackup usage message:

Usage: purgeFiles [OPTION]...
-h, --help                          Print this help message
-a, --ages=age1,age2                Desired ages to keep (in days)
-d, --directory=dir                 Target directory
-p, --pattern=pattern               File pattern to match
-f, --force                         Force deletion (no simulation mode)

e.g. purgeFiles --ages=1,2,4,40 --directory=/tmp --pattern="*.txt"
This would purge /tmp and try to keep a files ending in .txt of 40 days, 4 days, 2 days and 1 day old.
Note: this would only do a simulation run. Specify --force to actually delete the files.

Author: John Quinn, http://johnandcailin.com/john

purgeBackup - the source

06 nov 2012 -- moved the source to gitHub since i still get a lot of suggestions for changes. have at it. https://github.com/doofdoofsf/purgeFiles

Hi guys from spain thanks

Hi guys from spain

thanks for sharing this useful tool! Will be awesome for my backup policiy :)

Thanks

cool Script, thanks. Keep

cool Script, thanks.

Keep care to include "0" as backup-days when starting this script _after_ a backup was made - eg if the previous nightly run was not successful.

Thank you! This is exactly

Thank you! This is exactly what I needed, and more.

Glad you found it useful

Glad you found it useful

Useful Unix Utility of the

Useful Unix Utility of the Day finally paid off, eh JQ ? :-)

Lol. Indeed it did :)

Lol. Indeed it did :)

post new comment

the content of this field is kept private and will not be shown publicly.
  • web page addresses and e-mail addresses turn into links automatically.
  • allowed html tags: <h2> <h3> <h4> <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • lines and paragraphs break automatically.
  • you may post code using <code>...</code> (generic) or <?php ... ?> (highlighted php) tags.

more information about formatting options

captcha
are you human? we hope so.
copy the characters (respecting upper/lower case) from the image.