Fedora cluster project? (was Re: [Beowulf] Opteron/Athlon Clustering)

[Greg Kurtzer]

On Thu, Jun 17, 2004 at 07:25:20PM -0400, Jeffrey B. Layton mentioned:
> >>Up to date is not always good (I've been there before :)
> >>   
> >>
> >
> >Agreed; on the other hand, if cAos-2 will be "much more advanced", then
> >aren't you/they moving a little in the more up-to-date direction as
> >well?
> > 
> >
> 
> Yes, but not like FC. FC is moving on the bleeding edge (but perhaps
> not as much as Gentoo). I like the concept of having a distro "out there",
> but not for production clusters. cAos-2 will have some "advanced"
> features, but not nearly as agressive as FC2, but definitely more
> aggressive that straight RHEL. I don't know where one might put
> it on the distro "spectrum" from RHEL on one end and FC2 on the
> other end.

cAos has a different model for keeping things current then other distros do
in practice. We have a stable core. The core is a self hosting minimal
development environment. Our core consists of about 75 packages, and that is
maintained by the cAos core development team. The rest of the packages are
ports that sit ontop of this stable core thus making them extended pakcages...
The extended OS is independent of the core WRT to updates and bugfixes. The
core is what we are guaranteeing stability on. The extended packages can
remain bleeding edge without it affecting the underlying OS, or non-related
packages.

This also allows us to very successfully distribute the workload between many
maintainers, keep a stable core without affecting the extended packages, and
allow minimal installs to be,... well... minimal! :) We are keeping close
track of package requirements, so gigantic dependency trees are not created
for basic package installs. From my perspective, this is important for
maintaining cluster node file systems.

> >>I really don't like anaconda. Have you looked at the code - yuck! The
> >>installed in cAos (cinch) is a much better idea. It's not GUI, but that
> >>is a good thing IMHO. You can even hack it since it's just a bash script!
> >>   
> >>
> >
> >IMHO, anaconda can be as ugly as it wants, as long as red hat keeps
> >maintaining it and making sure it runs on a ridiculous variety of
> >hardware, which they appear to be doing.  Was running clean code the
> >principal reason for developing cinch, or was there functionality that
> >didn't exist in anaconda that the cAos developers wanted?
> > 
> >
> 
> Not sure, maybe Greg will pick up on this one.

The issue was that we needed an installer. I looked into Anaconda, but didn't 
have the time or resources to deal with it. Also, on a general note, I felt 
that installers didn't have to be so complex and huge. RH pays >=1 FTE for 
maintenance of just anaconda! For an application that you only use once 
(hopefully) per system, that is a lot of development effort!

I don't consider Cinch to be the greatest or most functional installer, but it
gets the job done with minimal maintenance.

> However, I would hazard to guess that it was because of clean
> code and scriptability (is that a word?).

Add simplicity as a major driving force. (actually this is also a driving
force behind cAos as well)

> >I think there's a lot to like about the warewulf approach, if you have a
> >homogenous hardware environment; I seem to recall seeing heterogenous
> >support being one of the goals for ww2 and I'm looking forward to that. 
> >We're not heterogenous but I think there's a decent chance that we'll
> >end up that way.  
> >
> 
> I'm doing heterogenuous clusters now. It's not difficult. I don't
> know about mixing I2's, Opterons, PIII, etc. But you can
> easily mix 32-bit processors of various models and makes. I'll
> have to bug Greg about that one, but I don't think mixing
> such a wide range of architectures is a wise idea (depending upon
> your problem). So, you can easily do it today but not on a weird
> "franken-cluster" (even though we all probably have one of those
> in our basement).

Warewulf-2.2 easily supports multiple VNFS's and kernels. There is nothing
that would stop it from maintaining nodes of ia32, x86_64, and ia64 in one
cluster, but I don't think I need to mention the levels of complexities that
introduces regarding binary portability. ... But it is possibly! hehe

> >In any case, if I do end up participating in a fedora
> >clustering project I'd certainly want to package up the warewulf vnfs
> >and tools as an option there.
> > 
> >

One of the first things that would have to happen would be for someone to
clean up the package requirements to avoid the massive dependency trees that
would occur with building minimal node file systems. After that, things get
easier... Unfortunately, the dependency trees in Fedora are getting more and
more complex from what I hear. :(

I am happy to help where I can, but I don't run Fedora. hehe
-- 
Greg Kurtzer
Berkeley Lab, Linux guy