[Beowulf] automount on high ports

[Robert G. Brown]

On Wed, 2 Jul 2008, Perry E. Metzger wrote:

>
> "Robert G. Brown" <rgb at phy.duke.edu> writes:
>> On Wed, 2 Jul 2008, Perry E. Metzger wrote:
>>> By the way, you can now design daemons to handle tens of thousands of
>>> simultaneous connections with clean event driven design on a modern
>>> multiprocessor with plenty of memory. This is way off topic, though.
>>
>> Not on a cluster list.
>
> Well, it actually kind of is. Typically, a box in an HPC cluster is
> running stuff that's compute bound and who's primary job isn't serving
> vast numbers of teeny high latency requests. That's much more what a
> web server does. However...

I'd have to disagree.  On some clusters, that is quite true.  On others,
it is very much not true, and whole markets of specialized network
hardware that can manage vast numbers of teeny communications requests
with acceptably low latency have come into being.  And in between, there
is, well, between, and TCP/IP at gigabit speeds is at least a contender
for ways to fill it.

>> I've written forking daemons (which is why I should have known, or
>> remembered, about the four-tuple thing:-) because they are an essential
>> component of IPCs in a network-based cluster or cluster distributed
>> apps.
>
> One is best off *not* forking, actually. There's a good site on
> concurrency management for high performance servers. It is a bit old
> now but covers the topic well: http://www.kegel.com/c10k.html
>
> Myself, I'm a believer in event driven code. One thread, one core. All
> other concurrency management should be handled by events, not by
> multiple threads. Thread context switching is very very expensive, and
> threads are very expensive. Doing event driven programming wins
> overwhelmingly in such contexts.  It is hard to impossible, on a
> modern machine, to handle tens of thousands of connections with
> forking or threads, but it is easy with events.
>
> I'm a fan of Niels Provos' "libevent" for such purposes. There are a
> lot of other libraries that plug in to it well, too.

Interesting.  Makes sense, but a lot of boilerplate code for daemons has
always used the fork approach.  Of course, things were "smaller" back
when the approach was dominant.  The forking approach is easy to program
and reminiscent of pipe code and so on.

     rgb

-- 
Robert G. Brown                            Phone(cell): 1-919-280-8443
Duke University Physics Dept, Box 90305
Durham, N.C. 27708-0305
Web: http://www.phy.duke.edu/~rgb
Book of Lilith Website: http://www.phy.duke.edu/~rgb/Lilith/Lilith.php
Lulu Bookstore: http://stores.lulu.com/store.php?fAcctID=877977