[Beowulf] General cluster management tools - Re: Southampton engineers a Raspberry Pi Supercomputer

[Prentice Bisbal]

On 09/12/2012 07:52 PM, Mark Hahn wrote:
> for the record, setting up ldap is trivial.  actually, configuring
> a whole cluster with stateless nodes is pretty straight checklist...
Yes and no. It's easy to you and me because we're professional system 
administrators who have been doing this for years. However, we talking 
about a class on building clusters that's for students, may have little 
or know system administration experience. Setting up a stateless cluster 
is more difficult than setting up a stateful cluster, there are more 
issues to worry about (DHCP, network booting, etc.)
>> I'd really like to know what challenges people are facing in this area.
>> Specific pain points.
> funding.  vendor lockin/licensing.
> lack of design standard for water cooling.
> 10G switches that freeze under load.
>
> installing and running clusters is easy.  it's the other stuff that's hard.
I have to agree here. For an experienced system admin, building and 
running a basic cluster isn't too hard, but the devil is in the details. 
My biggest problems have always been people and politics. Some examples:

- Management who doesn't understand clusters, or takes the vendors 
recommendations over the in-house expert(s)
- Vendors who try to sell you what they have, instead of what you need 
("Infiniband really isn't any better ethernet", or "You don't need a 
parallel filesystem. Our network attached storage device has plenty of 
power performance")
- Getting others to understand the importance of adequate power and 
cooling in the data center. A cluster is useless if you have to shut it 
down periodically because the datacenter is overheating.
- Explaining to users that they can't run commercial software package X 
on the cluster because there's no volume discount and vendor charges too 
much per node or per instance buy enough licenses. Ohhh.. and their 
department refused to contribute to the cluster budget.
- And then there's the difficult users...

>
>> <plug>Its Bright Cluster Manager. No, I dont work for them but they did
>> give me free licences. Yes its pretty good :)</plug>
> point-and-click is always being sold like crack: first hit is free ;)
>

I've found point-and-click works until you want to change something to 
suit your environment. Then you have to start customizing things, and 
that can get messy.

--
Prentice