ChunkSync

News

2017-10-24

Version 0.4 released. Adapted to be compatible with OpenSSL 1.1. Also, some minor code improvements.

2011-02-04

Version 0.3 released. Use more efficient (Linux) splice() syscall for moving data around, especially for the chunks themselves. No bugfixes.

2009-11-08

Version 0.2 released. Adds option -q for suppression of progress indication, useful for calling ChunkSync from cron. No bugfixes.

What is ChunkSync?

ChunkSync allows you to create space-efficient incremental backups of large files or block devices (encrypted disks, in particular) by splitting the data into a directory structure of chunk files which get hard-linked into new backup generations in case the contents of the respective chunk haven't changed (as judged by a SHA1 sum of the contents). This is similar to the way rsync's --link-dest option works, but a lot faster than using rsync on ChunkFS.

In case of remote sources and/or destinations, ssh is used for invoking ChunkSync backends on the remote machines.

The chunks themselves contain the bare backup data, so the original file's/device's contents can be restored by simply concatenating all the chunks from a backup tree. The layout of the tree is ChunkFS compatible, though, so the image can also be reconstructed using UnChunkFS, which is handy for restoring single files from a large filesystem image without first having to copy huge amounts of data, for example.

For more details, see the man page.

What Does it Run on?

It's been written for Linux, but I guess it should be easy to port to (other) Unix systems.

Download

Version Source Debian package i386 PGP signature
0.1 chunksync_0.1.tar.gz (21 KiB) chunksync_0.1_i386.deb (21 KiB) chunksync_0.1.asc
0.2 chunksync_0.2.tar.gz (22 KiB) chunksync_0.2_i386.deb (21 KiB) chunksync_0.2.asc
0.3 chunksync_0.3.tar.gz (22 KiB) chunksync_0.3_i386.deb (21 KiB) chunksync_0.3.asc
0.4 chunksync_0.4.tar.gz (22 KiB) chunksync_0.4_i386.deb (22 KiB) chunksync_0.4.asc

Feedback

In case you were wondering how to reach me: Florian Zumbiehl <florz@florz.de>, PGP key AF260AB1. Usually, you find me as florz on Freenode and OFTC, too. <xmpp:florz@florz.de> should do, also.

This paragraph is here just to get the following page linked into the public web for now, feel free to ignore it ;-): A LYME_SFX extraction perl script.