[Beowulf] RAID storage: Vendor vs. parts
canon at nersc.gov
Thu Mar 3 10:28:02 PST 2005
-----BEGIN PGP SIGNED MESSAGE-----
This was my experience as well. Most of the tools worked out of the
box, but the partitioning was a real hang up. The easiest way around
this is to have a separate boot drive that the installer can partition
in its normal manner and then to have the >2TB device be completely
separate and configured after first boot. You then use the whole drive
(no partitions). This worked for me.
Paul Nowoczynski wrote:
| Alvin Oga wrote:
|> hi ya steve
|> On Tue, 22 Feb 2005, Steve Cousins wrote:
|>> That's what I'm shooting for. Anybody have good luck with volumes
|>> than 2 TB with Linux? I think LSI SCSI cards are needed (?) and the 2.6
|>> Kernel is needed with CONFIG_LBD=y. Any hints or notes about doing this
|>> would be greatly appreciated. Google has not been much of a friend on
|>> this unfortunatlely. I'm guessing I'd run into NFS limits too.
|> for files/volumes over 2TB ... it's a question of libs, apps and
|> kernel everything has to work ... which is not always the case
| We've got this working at PSC without too much pain.. even with scsi
| block devices >2TB. The LBD is needed but it
| doesn't solve all the problems with large disks, especially if you have
| a single volume which is larger than
| 2TB. The issue we ran into was that many disk related apps like mdadm
| and [s]fdisk don't support
| the BLKGETSIZE64 ioctl. So even though your kernel is using 64 bits,
| some needed apps are not. There are also issues with disklabels for
| devices >2TB. The normal dos-style disklabel used by linux
| doesn't support them so you'll need a kernel patch for the "plaintext"
| partition table made by Andries Brouwer.
| If you're interested in running this on 2.6 I can give you the patch.
| As far as cards go I think the adaptec u320 cards
| are better. I've seen less scsi timeout weirdness with them (this could
| be related to our disks). Performance wise
| the lsi and adaptec are about the same.. we see ~400MB/sec when using
| both channels - even with a sub pci-x bus. For a couple hundred bucks a
| card this is really good news.
|> i don't play much with 2.6 kernels other than on suse-9.x boxes
|>> Also, am I being overly cautious about having a spare RAID controller on
|>> hand? How frequent do RAID controllers go bad compared to disks, power
|>> supplies and fan modules? I'd guess that it would be very infrequent.
|> it's always better to have spare parts ... ( part of my requirement ) if
|> they expect the systems to be available 24x7 ...
|> - more importantly, how long can they wait, when silly inexpensive
|> things die, before it gets replaced
|> - dead fans is $2.oo - $15 each to keep the disks cool
|> - power supply is $50 range ... but if one bought n+1 powersupply
|> than its supposed to not be an issue anymore, but you will need to
|> have its replacement handy
|> - raid controllers should NOT die, nor cpu, mem, mb, nic, etc
|> and it's not cheap to have these items floating around as spare
|> - ethernet cables will go funky if random people have access
|> to the patch panels ... ( keep the fingers away )
|> - ups will go bonkers too
|> - what failure mode can one protect against and what will happen
|> if "it" dies
|> - best protection against downtime for users is to have an
|> warm-swap server which is updated a hourly or daily ... ( my
|> preference - 2nd identical or bigger-disk capacity system )
|>> Looking back at my own experience I think I've had to return one out
|>> of 15
|>> in the last eight years, and that was bad as soon as I bought it.
|> seems too high of a return rate ?? 1 out of 15 ??
|>> If this is too off-topic let me know and I'll move it elsewhere.
|> ditto here
|> 24x7x365 uptime compute environment is fun/frustrating stuff on tight
|> c ya
|> Beowulf mailing list, Beowulf at beowulf.org
|> To change your subscription (digest mode or unsubscribe) visit
| Beowulf mailing list, Beowulf at beowulf.org
| To change your subscription (digest mode or unsubscribe) visit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
-----END PGP SIGNATURE-----
More information about the Beowulf