[Beowulf] recommendation on crash cart for a cluster room: fullcluster KVM is not an option I suppose?

[Mark Hahn]

> Even with IPMI, you still need a crash cart of some type to initially
> set up IPMI in the system's BIOS. At the minimum, you need to set the IP
> address that the IMPI interface will listen on (if it's a shared NIC

afaik, not really.  here's what I prefer: cluster nodes normally come out 
of the box with BIOS configured to try booting over the net before local HD.
sometimes this is conditional on the local HD having no active partition.

great: so they boot from a special PXE image I set up as a catchall.
(dhcpd lets you define a catchall for any not nodes which lack a their own
MAC-specific stanza.)  when nodes are in that state, I like to auto-configure
the cluster's knowlege of them: collect MAC, add to dhcpd.conf, etc.  at 
this stage, you can also use local (open) ipmi on the node itself to
configure the IPMI LAN interface:
 	ipmitool lan 2 set password pa55word
 	ipmitool lan 2 set defgw ipaddr 10.10.10.254
 	ipmitool lan 2 set ipsrc dhcp

none of this precludes tricks like frobing the switch to find the port-MAC
mappings of course - the point is simply that if you let unconfigured nodes
autoboot into a useful image, that image can help you automate more of the
config.

for a while I had a sort of 'borg' cluster that would autoconfigure
anything that PXEd on its LAN.  (well, assuming it would boot at least
an ia32 image - that image would add it to the cluster and arrange for
it to reboot into a more specific (eg x86_64) image.)  I never even 
bothered to mess with the BIOS boot-order on those nodes - they would have
tried to boot from the local disk before PXE, except that I left it 
unpartitioned.  local filesystem on /def/hda (ie, not hda1).  swapfile.