[Beowulf] OT: recoverable optical media archive format?

[Tomasz Rola]

On Tue, 8 Jun 2010, David Mathog wrote:

> This is off topic so I will try to keep it short:  is there an
> "archival" format for large binary files which contains enough error
> correction to that all original data may be recovered even if there is a
> little data loss in the storage media?  
> 
> For my purposes these are disk images, sometimes .tar.gz, other times
> gunzip -c of dd dumps of whole partitions which have been "cleared" by
> filling the empty space with one big file full of zero, and then that
> file deleted.  I'm thinking of putting this information on DVD's (only
> need to keep it for a few years at a time) but I don't trust that media
> not to lose a sector here or there - having watched far too many
> scratched DVD movies with playback problems.
> 
> Unlike an SDLT with a bad section, the good parts of a DVD are still
> readable when there is a bad block (using dd or ddrescue) but of course
> even a single missing chunk makes it impossible to decompress a .gz file
> correctly.  So what I'm looking for is some sort of .img.gz.ecc format,
> where the .ecc puts in enough redundant information to recover the
> underlying img.gz even when sectors or data are missing.   If no such
> tool/format exists then two copies should be enough to recover all of an
> .img.gz so long as the same data wasn't lost on both media, and if bad
> DVD sectors always come back as "failed read", never ever showing up as
> a good read but actually containing bad data.  Perhaps the frame
> checksum on a DVD is enough to guarantee that?

I use tar, gzip/bzip2, split - for creating a number of files of more or 
less similar lenghts (like, 50 megs or 100 megs, but usually 50).

After that, I make par2 recovery files with par2cmdline tools (they make
use of Solomon-Reed error correction)

http://en.wikipedia.org/wiki/Parchive
http://parchive.sourceforge.net/

I am unable to find par2cmdline via google ATM, but they should be 
somewhere.

And last but not least, I burn it all (data + pars). HTH.

Regards,
Tomasz Rola

--
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola at bigfoot.com             **