[Beowulf] OT: recoverable optical media archive format?

[David N. Lombard]

On Thu, Jun 10, 2010 at 12:20:39PM -0700, David Mathog wrote:
> Jesse Becker and others suggested:
> 
> >     http://users.softlab.ntua.gr/~ttsiod/rsbep.html
> 
> I tried it and it works, mostly, but definitely has some warts.
> 
> To start with I gave it a negative control - a file so badly corrupted
> it should NOT have been able to recover it.
> 
> % ssh remotePC 'dd if=/dev/sda1 bs=8192' >img.orig
> % cat img.orig      | bzip2 >img.bz2.orig
> % cat img.bz2.orig  | rsbep > img.bz2.rsbep
> % cat img.bz2.rsbep | pockmark -maxgap 100000 -maxrun 10000
> >img.bz2.rsbep.pox
> % cat img.bz2.rsbep.pox | rsbep -d -v >img.bz2.restored
> rsbep: number of corrected failures   : 9725096
> rsbep: number of uncorrectable blocks : 0
> 
> img.orig is a Windows XP partition with all empty space filled with
> 0x0 bytes.  That is then compressed with bzip2, then run
> through rsbep (the one from the link above), then corrupted
> with pockmark.  Pockmark is my own little concoction, when used as
> shown  it stamps 0x0 bytes starting randomly every (1-MAXGAP) bytes, for
> a run of (1-MAXRUN).  In both cases the gap and run length are chosen at
> random from those ranges for each new gap/run.

The website is more interested in corrupted block media, with the assumption
said corruption manifests as a cluster of invalid blocks from the file.  You've
got a different type of corruption.

> % cat img.bz2.rsbep | pockmark -maxgap 1000000 -maxrun 10000
> >img.bz2.rsbep.pox2
> % cat img.bz2.rsbep.pox2 | rsbep -d -v >img.bz2.restored2
> rsbep: number of corrected failures   : 46025036
> rsbep: number of uncorrectable blocks : 0
> % bunzip2 img.bz2.restored2
> bunzip2: Can't guess original name for img.bz2.restored2 -- using
> img.bz2.restored2.out
> bunzip2: img.bz2.restored2: trailing garbage after EOF ignored
> % md5sum img.bz2.restored2.out img.orig
> 7fbaec7143c3a17a31295a803641aa3c  img.bz2.restored2.out
> 7fbaec7143c3a17a31295a803641aa3c  img.orig

He documents a "freeze.sh" and "melt.sh" (in the contrib dir) that wrap rsbep
and rsbep_chopper.  That's very different from what you did.

[dnl at closter ~]$ ls -l junk.avi
-rw-rw-r-- 1 dnl dnl 12622344 2010-02-28 19:17 junk.avi
[dnl at closter ~]$ freeze junk.avi  > freeze1
[dnl at closter ~]$ melt freeze1 > melt1
[dnl at closter ~]$ md5sum freeze1 junk.avi melt1 
4f8052c358e5bd86b9bfffd980726940  junk.avi
dcbeafa75ec60f50d003876866009213  freeze1
4f8052c358e5bd86b9bfffd980726940  melt1
[dnl at closter ~]$ ls -l junk.avi freeze1 melt1 
-rw-rw-r-- 1 dnl dnl 14565600 2010-06-10 14:56 freeze1
-rw-rw-r-- 1 dnl dnl 12622344 2010-02-28 19:17 junk.avi
-rw-rw-r-- 1 dnl dnl 12622344 2010-06-10 14:56 melt1
[dnl at closter ~]$ 

The above only shows a single example of it not damaging an intact file.
But I only played with it for about 10m last night and the above is a
laugh test.

I'll start proper testing tonight...

-- 
David N. Lombard, Intel, Irvine, CA
I do not speak for Intel Corporation; all comments are strictly my own.