[Beowulf] best archetecture / tradeoffs

[Robert G. Brown]

David S. writes:

> A clever way I've seen of doing NFS-root uses AFS instead, with a
> replicated root volume mounted read-only on the clients.  AFS' local
> caching makes it fairly efficient, and replicating the volume means
> the server is no longer a single point of failure.  As long as one
> server hosting the volume is available, the clients will keep running.
> I've considered using AFS in place of NFS in clusters, but have been
> been discouraged by the diffculty of managing tokens and Kerberos 
> tickets has discouraged.  AFS-root is pretty slick, though.
> 
> David S.

Hahahahaha...

This is a joke reply, right:-)

Let's see.  Yes, there are tokens, which often expire mid-computation.
There is kerberos, a lot of overhead indeed in a firewalled private
internal network.  There is AFS's "I didn't really mean it" attitude
towards implementing e.g. fflush on mounted AFS volumes, where writeback
occurs only when a file is closed (making it impossible to use shared
files across nodes for a variety of purposes without adding all sorts of
file stats and overhead).

The one think I do like about AFS is its "real" ACLs.  Unix's file perms
suck as a mechanism for enabling shared/group work, although they do
give sysadmins that warm fuzzy feeling of approximate control.

Seriously, we've kicked AFS around as a cluster FS before here, and I've
even used it in some computations I did a decade or so ago (where I
learned the hard way about the hahaha attitude towards implementing
fflush()) but it doesn't seem to be robust or at all easy to manage for
this sort of thing -- overkill in one place (security), underkill
someplace else (behaving like a rational/reliable filesystem).

Although RO AFS -- that one I haven't heard of, and obviously fflush
problems are irrelevant.  So it might not be a joke after all:-) Don't
you need a pretty special kernel to make that work?

    rgb
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://www.scyld.com/pipermail/beowulf/attachments/20050830/3ccb39c5/attachment.bin