[Beowulf] Should I go for diskless or not?

[Joe Landman]

John Hearns wrote:
> 2009/5/15 Joe Landman <landman at scalableinformatics.com>:
>>>>  Swap is an issue.  A somewhat hard to solve issue ... we'd recommend
>> actually turning off swap (and swappiness in the kernel) for diskless.  Or
>> put a USB drive in each machine and swap on that, though, honestly, that is
>> as reliable as swapping over the network. E.g. don't do it.
> 
> At the risk of boring everyone here rigid with the "SGI" word, ICE
> clusters mount swap via iSCSI.
> It seems to work - I say seems, as I've never actually looked at a
> blade doing swap in anger.

We have tried this on other units, pushing the envelope hard on swap. 
What we find is, contrary to almost every distros vendors claims, 
OOM-killer wakes up and goes all wild-west on you.

You never, under any circumstances, want to wake OOM-killer.  You want 
to back away slowly.

I see more crashed nodes with "oom" collateral damage in the logs, than 
I care to admit.

> Which reminds me - I have a to-do list task which is to test the OOM
> killer on one of these blades.
> Guess I can look at it swapping at the same time as I stress it for OOM.

Oh ... a glutton for punishment ... eek!

> 
> Two rules of thumb for swap though:
> 
> a) provision enough RAM on ya nodes for ya code

always.

> 
> b) hire Samuel L Jackson to get medieval on the ass of any users dumb
> enough to run code which leaks memory

ROTFLMAO!

I forgot which parody of Pulp Fiction this came from ... absolutely 
hilarious!  I think it was a MadTV skit.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
        http://jackrabbit.scalableinformatics.com
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615