[Beowulf] Should I go for diskless or not?

[Jan Heichler]

Hallo Dr,

Mittwoch, 13. Mai 2009, meintest Du:


I have a cluster of identical computers. We are planning to add more nodes later. I was thinking whether I should go the diskless nodes way or not? Diskless nodes seems as a really exciting, interesting and good option, however when I did it I needed to troubleshoot a lot. I did fix it up, but I had to redo the filesystem, but the past experiences didn't make much of a difference. I still need to fix up everything, I kinda need your help to decide.
Also, performance wise, I was thinking that diskless is not a good option, and since performance matters . . .
Can somebody outline the pros and cons of each or just give me thier opinion.


Local disk allows you to have 
 - local cached version of the OS-Image (could lead to faster bootup - depends on the image size)
 - local swap - can be used to suspend jobs and free the memory they are using by swapping it to disk. A newly started high-prio-job can then be started
 - local scratch - might be useful for some jobs
 - saves memory because you don't have to put some OS-image into RAM
 - avoids network trafic (no NFS-Root, no /usr-mounts over NFS or such stuff... )

Local disk cons:
 - it is a piece of hardware that can fail (might matter if you have a big number of nodes/disks)
 - it costs money

I saw lately that a customer was using a Lustre-Filesystem for scratching (no big news, can be much faster than local disks) and to put swap-files on it. Might be a good compromise - but just if you have a lustre-environment anyway. 

Jan                            
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.scyld.com/pipermail/beowulf/attachments/20090514/9ddf51ce/attachment.html