[Beowulf] 48-Core X86_64 Compute Node - Good Idea?

[Jon Forrest]

On 6/14/2010 10:29 AM, Mark Hahn wrote:
>> right now, I believe such boxes aren't available
>> yet. The closest thing is a 4-way 1U box, which
>> gives 48 cores per rack unit, but in *1 node*.
>
> well, the supermicro website lists them:
> http://www.supermicro.com/Aplus/system/2U/2022/AS-2022TG-HTRF.cfm
> http://www.supermicro.com/Aplus/system/2U/2022/AS-2022TG-HIBQRF.cfm

Those are both 2U boxes. I was hoping to find
a 1U box since they cost less to co-locate.

>> My intuition tells me that I should be wary of
>> such a configuration because of various SMP-related
>> locking and concurrency issues.
>
> why? is there something peculiar about your workload, and especially
> something that would show up with modestly higher SMPness?

Nothing specific. I'm worried about latency and locking
issues that only pop out when larger numbers of cores
are used.

> this is hardly uncharted territory. SGI's been there forever,
> and some fringe boxes from Intel. but 8s 4c has been pretty mundane
> for a while, and doesn't need any sort of hand-holding. unless you
> mean something like "I expect to swap a lot and want to configure a
> single non-raid swap partition", I don't really see what you're worrying
> about...

SGI isn't mainstream, and probably doesn't use the
same chipset and motherboards that SuperMicro
will be selling.

> I think people should actually take fresh look at 4s 1U boxes
> because AMD has eliminated the "4-socket penalty". there are some
> nontrivial advantages to fatter nodes - they let you achieve some
> unique workload configurations (bigger memory, higher-threaded, etc).
> sysadmin work doesn't scale linearly as the number of nodes, of course,
> but having fewer, fatter nodes can be attractive TCO-wise, too.

If SuperMicro doesn't come up with Twin boxes, I might
be forced to follow your advice. I'm not concerned
about sysadmin work, because I'm using Rocks. I'm more
concerned about ending up in the Twilight Zone where
things aren't as they appear.

-- 
Jon Forrest
Research Computing Support
College of Chemistry
173 Tan Hall
University of California Berkeley
Berkeley, CA
94720-1460
510-643-1032
jlforrest at berkeley.edu