[Beowulf] Is Beowulf the right solution for my problem ?

[Chris Dagdigian]

A beowulf will work; you just need to complement it with a "distributed 
resource management" layer such as Grid Engine which will schedule and 
farm out your non-parallel tasks to the other COW nodes. These sorts of 
systems are great for handling large volumes of serial tasks, especially 
the 'not so parallelized' applications.

Grid Engine really is (in my opinion) the best free product for this 
situation, especially for life science use cases and workflows.

If you have a political problem with this you may want to have a 
discussion about "what is more important: politics or doing research?" 
within your organization. Especially if the political problem is coming 
from a hardware vendor or IT supplier or something equally silly.

You can choose a less capable system (openpbs) or you can pay money for 
an excellent commercial solution (Platform LSF). Your call. I'd suggest 
researching Grid Engine further before you dismiss it as a possibility.

My $.02 of course!

-Chris



Nicolas GUIOT wrote:

> Hello everybody.
> 
> I was just looking for a cluster management solution, and found out that Beowulf could be a good solution.
> I then started reading this : http://www.phy.duke.edu/~rgb/Beowulf/beowulf_book/beowulf_book/index.html
> 
> And I'm now wondering if my applications were right designed to run on the beowulf cluster.
> 
>>From what I read so far, it seems beowulf is adapted to make ONE calculation on several nodes, but this is not my objective : 
> -I have a COW of about 20 to 40 computers, mostly differents, in terms of CPU, RAM, and so on
> - We run several applications (Theoritical Bio-chemistry), not especially designed for parallel processing
> 
> My goal is to have a master node to which everyone supplies his jobs, then the master dispatches the jobs to the different computers, according to criterias (CPU must be Intel, CPU must be over 2 GHz, need much RAM, etc....)
> 
> So the question is : is beowulf adapted to my needs, and if it's not, do you know any available (free) solution for me ? (I've already heard about Sun Grid Engine, but for "political" reasons....)
> 
> Thanks for your help
> 
> Nicolas GUIOT