[Beowulf] How Can Microsoft's HPC Server Succeed?

[Anand Vaidya]

On Fri, Apr 4, 2008 at 6:19 PM, Geoff Galitz <geoff at galitz.org> wrote:

> ....
>
> More to the point of the thread:  if we are talking about MS making
> greater
> inroads into the HPC market, then their most likely tactic is to convince
> the commercial app vendors to write and/or port their apps to the Windows
> platform.  I've seen a number of animation and visualization apps running
> on
> MS clusters.  They do exist and this is their best near-term chance of
> making those inroads.  ......
> ....



>
> I've been doing clusters for approx 10 years, but like others I am now
> running Windows on my main workstation because it is the right combination
> of development environment, application availability and usability for me.
> Prior to Vista, this would not have been the case... so life and
> technology
> do evolve.
>

My company is  primarily a Linux / HPC shop. However, we have some contacts
with MS and have done a couple of WCCS installations.

When a vanilla Linux+SGE+xcat or ROCKS cluster (say from a CentOS disk) is
compared against WCCS, Linux is unbeatable: Price vs Features+Performance.
Just use the license fee payable to MS to pay to a service provider  and get
100% warm body support.

I heard MS telling that they are not really out to get the National Labs
kind of computing. They are more focussed on, atleast right now in this
region (ASEAN) , smallish (4-16 nodes) clusters running vendor certified
stack such as Fluent, Digital Rendering, calculations with an Excel UI  etc.


The strengths they extol is what you would suspect: AD integration, one
click job launches from your desktop, seamless file drag-and-drop, Office
integration (for the PHBs to check off ) , No relearning since it looks and
acts like a Windows machine (which is only partly true).

While I usually don't like the hairball that is MS software, MS has actually
put some effort and made a well integrated software (not bad for v1, WCCS
/2003). They bundle remote installation, optional local AD (on headnode), IB
is fully supported, a job scheduler that can be used via CLI, Win32 GUI and
via Webservices (potentially, Linux, & OSX can use this interface). The
whole thing is supported (of course, you pay for it).

Having said that, I think that the Linux clustering scene needs a little
competition, especially the for-fee ones. Apart from SDSC, not many
innovations are happening.  I am not referring to standalone projects, where
FOSS community has a lot of innovatio happening,  but rather one integrated
Linux Cluster  on a DVD that gets you a cluster ready in 20mins, with no
pain at all. ROCKS comes with its own problems, esp, wrt updates (which is
why I stopped using ROCKS), however they are working on this one, AFAIK.

Have a look at what RH and Novell offer as "cluster license". I will pick on
RH since that is what I am using for clustering: It is just a RHEL ES, plus
RHEL WS licenses.  No extra clustering stuff are packaged (eg: SGE, Ganglia)
nor are there any extra toolkits for managing the cluster ( reinstall,
status, cluster wide command execution, queue management etc). If I install
SGE on a RHEL cluster, RH is not going to support any problems with SGE.

For an admin who buys such a package, especially if (s)he is unfamiliar with
Linux, it is going to be a nightmare.

So, here's what the FOSS community, especially,  vendors (RH, Novell) should
be doing, specifically for a HPC oriented version:

- remove all unwanted packaged (desktop software, multimedia, web browsers
etc)
- package SGE, Ganglia,
- a good clustering toolkit, maybe derived from ROCKS scripts (I am biased
towards IBM xcat, because that is the only tool I use)
- LDAP as the default auth source, setup SSH for clusterwide passwordless
logins by default
- easy integration to AD
- provide default environments , SGE queues etc) or easy to use scripts/gui
- Many users have low-complexity jobscripts. Provide a web UI (jetspeed2,
tomcat ?). Let the advanced users tap the power of command line.
- package a selection of top20 FLOSS science apps (Gromacs, Phylip, Blast,
MPICH,  fasta, fftw etc)
- package and provide one click installation for restricted-ware such as
NAMD, or commercial software such as Intel Compilers, Fluent, Amber etc. It
CAN be done, Ubuntu has demonstrated how to do it well.
- package and provide easy install of a parallel filesystem such as GFS or
Lustre

My $0.02 + 7% Taxes

Regards
Anand




>
> Just my two cents.
>
> -geoff
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.scyld.com/pipermail/beowulf/attachments/20080405/6ac8239d/attachment.html