[Beowulf] Should I go for diskless or not?
deadline at eadline.org
Fri May 15 05:26:36 PDT 2009
You will note that I used sufficient wiggle words "usually" and
"generally" because in my experience it always depends.
And of course my comments are from my personal
experience. I have found that diskless allows for
the entire cluster to be "re provisioned" without
have into re-image disks. Reboots are quicker
(for the hardware I use) and since I use ram
disk approach (Warewulf/Perceus) I find that things
are a bit faster, also the diskless image has
minimal services running vs a disk-full distribution image.
(I fully understand the good admin can trim a
There are plenty of arguments either way. Back in 2006
I did a mini-poll on node disk space usage:
So "in general, it varies". YMMV
> Doug: Diskless provisioning is usually easier to manage.
> Hmm, not sure I buy that one. Pretty much any decent cluster distribution
> * allow you to add a compute node without much more than plugging it in
> and telling it to PXE boot (diskless or diskfull)
> * Allow you to push a configuration cluster wide
> * allow you to reinstall/reboot all nodes.
> Sure installing 100 diskfull nodes takes more network bandwidth than
> 100 diskless nodes. The flip side is booting 100 diskless nodes takes
> bandwidth then 100 disk nodes. For practical uses of clusters either way
> booting/installation is approximately 0% of the annual network bandwidth.
> Certainly installing 1000 nodes from a single fileserver would take quite
> awhile, but various technologies (bit-torrent and broadcast) remove that
> bottleneck. I'm kinda curious since diskless nodes don't really install
> do they handle heterogeneous hardware. Say a dead motherboard comes back
> a new pci-id or 3?
> Doug: In general diskless is faster.
> At what? Diskless booting is faster than diskfull booting or diskfull
> installation. Application performance should be pretty much the same.
> worst case scenarios like installing 1000 compute nodes from the head node
> usually dealt with by using broadcasts or bit-torrent.
> So it depends, my current thinking is that it's not worth the man hours to
> it yourself unless you have a larger cluster. If it's supported by your
> cluster distribution it could easily be worth it. In the whole scheme of
> things I'd worry about diskless last. Decide on your cluster distribution
> based on your application and user needs, systems administrator
> and budget.
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
More information about the Beowulf