[Beowulf] network filesystem

[Joe Landman]

Bruce Allen wrote:

> (1) RAID-across-nodes.  For example every ten nodes form a redundant 
> RAID set.  The disappearance of any one of these nodes causes no data 
> loss, service loss, or corruption at the user level.  The total 
> redundant storage available from the ten nodes is 90% of the available 
> raw storage.

Using iSCSI targets and iSCSI initiators, you could build RAID5 or RAID6 
across boxes using the linux MD device.  We have proposed this to some 
financial customers using our JackRabbit unit.

> (2) Symmetry: all nodes have identical behavior and features.  There are 
> no specialized IO or metadata nodes, which act as filesystem bottlenecks 
> and which are single points of failure.

For this, we use a set/pair/triple of thin HA servers with stonith 
running.  You can run them in active-passive, active-active (requires 
some sort of CFS then).  The nice part about this is that the metadata 
resides within the FS, and you look at each machine as a big block o disks.

> Am I correct that Lustre does not offer either of these features?

Lustre is an object data storage system.  Breaks apart metadata from data.

> 
> Do you (or does someone else) know if there is an open-source or 
> commercial distributed (posix) filesystem with these features?

If you use our idea above (iSCSI targets/initiators), you could run 
active-passive/STONITH mode using xfs/jfs .  We have proposed this at a 
number of places when they need very fast cutover, and downtime of any 
sort means significant loss.


> 
> Cheers,
>     Bruce

Joe

-- 

Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452 or +1 866 888 3112
cell : +1 734 612 4615