[Beowulf] A Good Linux Distribution to Start with?

[Tim Mattox]

Hello Jeff,

What processors/hardware is in your nodes?
Are they AMD64?  Do you have SATA drives?
A primary consideration is which distribution supports all your
hardware.  SATA support is still tough to come by.  Also full
AMD64 support is more wide spread now, but you still need to
check first if a distribution will do 64-bit mode "right out of
the box" as well as have both 32-bit and 64-bit dynamic
libraries.

I would recommend a Linux distribution that is supported by the
cluster management tool(s) that you select.  Some of these
tools will even supply the distribution for you.  The days of
installing a full non-cluster linux distribution on all your
nodes are gone, or at least numbered...

Take a look at any of these tools (in alphabetical order):
Clustermatic  http://www.clustermatic.org/
Oscar         http://oscar.openclustergroup.org/
Rocks         http://www.rocksclusters.org/
Warewulf      http://warewulf-cluster.org/

There are several commercial cluster distributions worth
considering as well.

I personally use Warewulf, and liked it so much I became one of
its developers.  Warewulf is a bit different in that it builds
on top of "most" RPM based distributions.  As one of its
developers, I can even foresee it eventually working with Debian
and/or Gentoo.  But currently it requires RPM and yum to do
it's work.

As for which distribution to use with it, I'm anxiously
awaiting the cAos-2 distribution to be ready from
http://caosity.org/
cAos-2 is built around the 2.6 kernel with gcc 3.4.x, and
will have full AMD64 support.  It has other good
features, but those are the ones I care about.  In the
meantime, I use cAos-1 with it's optional gcc3 package
and a 2.4.27 kernel.

The reasons why I use cAos-1 & 2 is that they are easily set up
in very minimal installs, yet are very easily added to if I
need additional packages.

The reason I use Warewulf is it's cluster management philosophy
is very flexible yet simple.  All nodes in the cluster boot
from a ramdisk image that is built on the master (or head)
node. That ramdisk image is easily customized, and updates to
the nodes are just a reboot or rsync away.

This is drifting off topic, so I'll leave with this thought: 
Rather than asking what distribution to use for your Beowulf,
you may want to ask what Beowulf toolset should you use...
-- 
Tim Mattox - tmattox at gmail.com - http://homepage.mac.com/tmattox/