[Beowulf] Any recommendations for a good JBOD?
landman at scalableinformatics.com
Fri Feb 19 18:52:47 PST 2010
Alex Chekholko wrote:
>> Thanks for the pointers! I had never heard of AoE before!
> This is all well and good until you compare the prices of the respective solutions.
> E.g. what's the cheapest 5TB (usable) AoE box you can buy?
I believe somewhat more than a relatively fast iSCSI/SRP/NFS/CIFS box
with 6.75TB usable (but we are biased).
AoE hasn't really found a niche for many reasons. Not the least of
which is the paucity of quality software target implementations, a
single vendor hardware supplier, and the significant resource consuming
initiator. We have lots of experience setting up AoE systems for users
and customers, lots of experience fixing systems, and altering designs
so that the AoE initiators don't bring down head nodes, login nodes, etc
that they are attached to. It is best, when building an AoE system
design, to isolate the unit mounting the AoE targets from important
services that have to be up.
Whether or not the AoE protocol is superior to iSCSI is moot. The AoE
initiator for windows isn't terribly stable. Last I checked there was
something for Solaris though I don't know the state. iSCSI is
everywhere, it is available on pretty much all platforms ... it has
achieved ubiquity. Quality targets and initiators exist as software
stacks you can use. They interoperate reasonably well, though the
Windows 2.08 stack doesn't seem to follow the standard terribly well in
terms of reconnecting to a single target. These minor nits aside, it
works, reasonably well, and without significant pain.
This aside, both AoE and iSCSI provide block device services. Both
systems can present a block device with a RAID backing store. Patrick
and others will talk about the beauty of the standards, but this is
unfortunately irrelevant in the market. The market isn't a meritocracy.
Actually, with the advent of USB3 and related devices, I'd expect AoE
and lower end raid to be effectively completely subsumed by this. USB3
has ample bandwidth to connect a low end RAID unit, more than GbE. The
comments on the Atom based micro itx MBs are quite relevant there.
Especially if you can get one with multiple SATA and USB3 ports
(especially if they can be USB3 targets).
If you want the lowest end RAID right now, get an eSATA device and
enclosure (not cheap but doable). It will work, albeit not as fast as
you might like. Use mdadm, and be done with it.
Remember, if you want to use mdadm, you do need block devices to work
with. Whether these block devices are gigabit, USB, or eSATA attached
is irrelevant. Better/more expensive systems will put the RAID on the
unit and export you either a block device, a file system, or both. Even
better systems will give you thin provisioning, snapshotting, and all
these other very nice features. You make your choices and pay your money.
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
More information about the Beowulf