home | projects | about

Sunday, December 19, 2004

Link-Backup v 0.4

I've released Link-Backup v 0.4 with a number of new capabilities, the primary being "incremental backup". Now you can specify the number of minutes to run with --minutes. If the backup doesn't complete in this time, it'll resume where it left off next time it is run again.

Instead of hard-linking between trees, now trees hard link to a central "catalog" of unique files. The catalog has none of the structure of the trees and in a way is simply a large md5(+file stats) to inode lookup table. This approach allows simple incremental backup: the catalog is updated independent from tree creation. Once the catalog update is complete, a tree is created that hardlinks to the catalog.

Other new features include being able to calculate a "diff" between the source and dest, and then a way to update the catalog of the dest with this diff. This is useful if you are doing remote backup and want to take a particularly large set of new files to the destination by hand (or by laptop). Sometimes my DSL can't keep up with the inflow of pictures!

Note: the v0.4 backup format isn't compatible with v0.3! Contact me if you want the v0.3 to v0.4 archive converter.

From the history:

v 0.4 11/14/2004 scottlu
- Changed a central catalog design with backup trees hardlinking to the catalog.
This way catalog updating can be incremental.
- Removed filemaps - not required any longer
- Changed logging to occur in the catalog as well as backups. Changed log parsing
methods accordingly
- Added incremental backup feature --minutes
- Make md5hash calculation incremental so a timeout doesn't waste time
- Created 0.3-0.4.py for 0.3 to 0.4 upgrading
- Added --showfiles, shows differences between src and dst
- Added --catalogonly, updates catalog only, doesn't create tree
- Added --filelist, specifies file list to use instead of tree
- Removed --rmempty
- Added --verbose