[Beowulf] RAID storage: Vendor vs. parts

[Shane Canon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


This was my experience as well.  Most of the tools worked out of the
box, but the partitioning was a real hang up.  The easiest way around
this is to have a separate boot drive that the installer can partition
in its normal manner and then to have the >2TB device be completely
separate and configured after first boot.  You then use the whole drive
(no partitions).  This worked for me.

- --Shane

Paul Nowoczynski wrote:
| Alvin Oga wrote:
|
|> hi ya steve
|>
|> On Tue, 22 Feb 2005, Steve Cousins wrote:
|>
|>
|>
|>> That's what I'm shooting for. Anybody have good luck with volumes
|>> greater
|>> than 2 TB with Linux?  I think LSI SCSI cards are needed (?) and the 2.6
|>> Kernel is needed with CONFIG_LBD=y.  Any hints or notes about doing this
|>> would be greatly appreciated.  Google has not been much of a friend on
|>> this unfortunatlely. I'm guessing I'd run into NFS limits too.
|>>
|>
|>
|> for files/volumes over 2TB ... it's a question of libs, apps and
|> kernel     everything has to work ... which is not always the case
|>
|>
|>
| We've got this working at PSC without too much pain.. even with scsi
| block devices >2TB.  The  LBD is needed but it
| doesn't solve all the problems with large disks, especially if you have
| a single volume which is larger than
| 2TB.  The issue we ran into was that many disk related apps like mdadm
| and [s]fdisk don't support
| the BLKGETSIZE64 ioctl.  So even though your kernel is using 64 bits,
| some needed apps are not. There are also issues with disklabels for
| devices >2TB.  The normal dos-style disklabel used by linux
| doesn't support them so you'll need a kernel patch for the "plaintext"
| partition table made by Andries Brouwer.
| If you're interested in running this on 2.6 I can give you the patch.
| As far as cards go I think the adaptec u320 cards
| are better.  I've seen less scsi timeout weirdness with them (this could
| be related to our disks).  Performance wise
| the lsi and adaptec are about the same.. we see ~400MB/sec when using
| both channels - even with a sub pci-x bus. For a couple hundred bucks a
| card this is really good news.
| --paul
|
|>     i don't play much with 2.6 kernels other than on suse-9.x boxes
|>
|>
|>
|>> Also, am I being overly cautious about having a spare RAID controller on
|>> hand?  How frequent do RAID controllers go bad compared to disks, power
|>> supplies and fan modules?  I'd guess that it would be very infrequent.
|>>
|>
|>
|> it's always better to have spare parts ... ( part of my requirement ) if
|> they expect the systems to be available 24x7 ...
|>     - more importantly, how long can they wait, when silly inexpensive
|>     things die, before it gets replaced
|>
|>     - dead fans is $2.oo - $15 each to keep the disks cool
|>
|>     - power supply is $50 range ... but if one bought n+1 powersupply
|>     than its supposed to not be an issue anymore, but you will need to
|>     have its replacement handy
|>
|>     - raid controllers should NOT die, nor cpu, mem, mb, nic, etc
|>     and it's not cheap to have these items floating around as spare
|>     parts
|>
|>     - ethernet cables will go funky if random people have access
|>     to the patch panels ... ( keep the fingers away )
|>
|>     - ups will go bonkers too
|>
|>     - what failure mode can one protect against and what will happen
|>     if "it" dies
|>     - best protection against downtime for users is to have an
|>     warm-swap server which is updated a hourly or daily ...     ( my
|> preference - 2nd identical or bigger-disk capacity system )
|>
|>
|>
|>> Looking back at my own experience I think I've had to return one out
|>> of 15
|>> in the last eight years, and that was bad as soon as I bought it.
|>>
|>
|>
|> seems too high of a return rate ?? 1 out of 15 ??
|>
|>
|>
|>> If this is too off-topic let me know and I'll move it elsewhere.
|>>
|>
|>
|> ditto here
|> 24x7x365 uptime compute environment is fun/frustrating stuff on tight
|> budgets
|>
|> c ya
|> alvin
|>
|> _______________________________________________
|> Beowulf mailing list, Beowulf at beowulf.org
|> To change your subscription (digest mode or unsubscribe) visit
|> http://www.beowulf.org/mailman/listinfo/beowulf
|>
|>
|
| _______________________________________________
| Beowulf mailing list, Beowulf at beowulf.org
| To change your subscription (digest mode or unsubscribe) visit
| http://www.beowulf.org/mailman/listinfo/beowulf
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCJ1cxZd/2zrI5CioRAnYnAJ98qtd17aPK62aCw4UNt79klUdasQCglLyo
kNXL0h7KGaQfFmla33Gxfn4=
=osvt
-----END PGP SIGNATURE-----