[Beowulf] Building new cluster - estimate
kyron at neuralbs.com
Wed Aug 6 18:33:10 PDT 2008
Joe Landman wrote:
> Eric Thibodeau wrote:
>> And select the main stuff (HDD drivers) as built in and don't fsck
>> around with the initrd stuff, that's only usefull for kernels that
>> need to be generic and adapt to all hardware (ie: install
>> CDs)...other than that, monolithic a kernel works fine ;)
> Advantage of modules is you can upgrade them without upgrading the
> kernel. Go ahead, build in that e1000 driver. I dare yah... :(
Ok...I didn't put enought emphasis on "main" stuff....as in, _all you
need to get the system booted, which essentially means HDD chipset
drivers, the rest I do build as a module (NIC, video and such).
> More to the point it does give some good flexibility for end users
> with a need to keep the core "separate" from the drivers for maintenance.
> Initrd is subtle and quick to anger. One must use burnt offerings to
> placate the spirits of initrd.
> Well, it would be a heck of a lot nicer if the tools were a little
> more forgiving ... Oh you don't have this driver in your initrd ... ok
> ... PANIC (mwahahahaha)
Pahahahahah... Point in case, I am building a CD-only cluster system
(based on Gentoo) and I am currently _NOT_ using initrd because all that
really needs to be built in is NFSroot support an all NICs I care to put
in. Obviously this is a deprecated approach but it's proven to be the
most effective and easy to maintain in my case.
>> ...and such. I'd tell you to use the Gentoo Clustering LiveCD but
>> that's work in progress...you could still build the cluster using
>> Gentoo...if you're performance savvy...and want things like OpenMP
>> capable compiler
> I have been hearing claims like this for a long time. I have not seen
> any real tests that back these claims up. Do you have any?
I'm actually working on such benchmarks. Did you know that compiling
with the default ICC optimization will cause your bridge to crumble due
to floating point assumptions?...
Ok, so my computation have diverged horribly mostly because I am
computing 47(vector size)*5000(K-Means clusters)*6,787,955(learning
dataset)*5(iterations to convergence) for a total of 7,975,847,125,000
FLOPS (or about 8Tera FLOPS) as part of an iterative learning process,
the error adds up. So performance is very sensitive to what your
intended goal is too ;)
> Most of the arguments I have heard are "oh but its compiled with
> -O3" or whatever. Any decent HPC code person will tell you that that
> is most definitely not a guaranteed way to a faster system ...
Hey...as I stated above, one would have to be quite silly to claim -O3
as the all well and all good optimization solution. At least you can
rest assured your solutions will add up correctly with GCC. To get a
"faster" system, you really have to look at your app, use strace, ltrace
and gprof, then you can play with that. What I _am_ saying though is
that Gentoo _does_ empower the administrator by giving him the ability
to customize the OS if a bottleneck is to be identified.
>> (gcc-4.3.1, or ICC ;) ) _integrated_ into your system (not a hackish
> Er... We often use several different compilers in several different
> trees. Several gccs, pgi, icc, eieio ... you name it. All are
Are-you currently able to run GCC-4.3.x versions on your current setup,
I'm actually eager to know. I'm still living under the ASSumption od
binary distributions not coping too well with multi-library
environments. Point in case, one of my colleagues _really_ wanted
firefox 3 on his ubuntu system. The installer trickled down to having to
uninstall glibc...and he forced it to YES (and this is just a browser,
not something that is used to _make_ code and would be tied to glibc)
>> afterthought of an RPM that pulls in a new glibc that breaks the install
> Er ... not the slightest clue as to what you are talking about. I
> haven't seen gcc, icc, pgi, ... touch our glibc.
> Maybe I am missing the fun. Which ICC version is this? Which gcc is
> this, which glibc is this?
Sorry about that I might have been misleading, GCC is generally the one
most sensitive to glibc, not the other ones although the latest ICC
(10.1.x series) do claim compatibility with the GNU environment so it
might get a little more dependency there.
More information about the Beowulf