four-tier applications (was re: WinNT/Linux)

[Ryan Sweet]

I'd like to take Richard Shilling's comments a step further and suggest
that the real solution for the scenario under discussion is a multi-tiered
application using browsers/MSOffice as the front end, combined with a
webserver, a database, and a compute cluster.  Ideally there should be
enough abstraction at each level that any component at another level
can be exchanged/upgraded/etc... without recoding.

In this scenario a compute job input is gathered on the website (or if you
must use MS Office _all the time_ then in an MSOffice Document with VBA
macros) and then is sent to a database server.  A batch scheduler for the
cluster then fetches its pending jobs from the database and runs them. The
cluster then sends its output or result sets back to the database, where
they can be retrieved by any number of web applications, including perl or
php or vb scripts that can convert the data into an MS Office friendly
format.  The tools for all of this already exist, they merely need to be
glued together.  This is what my company is doing for at least one of our
current cluster applications, though I'm attempting to guide the
development such that we can construct a framework that will be more
generally applicable.

I would propose an architecture like so:

* Client(win32) sets up job using a php driven website.
* Input from web presentation layer is sent to mysql running on a database
server.
* Cluster gateway node queries database server for input and initiates
job.
* The same database/webserver can be used for monitoring progress of the
job/performance of the cluster.
* The result set/output of the compute job is sent back to the database
(to a different table/tables, of course)
* The client browser uses the web presentation layer to query the result
set.
* A script initiated from the website can pull data from the database and
export to excel/access/word etc....

Depending upon the expected number of requests the database/website could
even be run from the gateway node itself, though for larger systems or
ones with many clients it makes sense to separate this layer.  

Now all of the above sounds nice, but the killer is that to implement it
_all_ one has to pull a bunch of different skills together
(high performance computing/beowulfing, database administration/design, web development, scripting, office automation) and
it costs _time_, but in the end I think it will make the difference
between systems that are academic in nature and ones that are truly
professional.

regards,
-Ryan Sweet, 
<rsweet at atos-group.nl>
Atos Netherlands, BV
http://www.atos-group.nl