Kidger's comments on Quadric's design and performance

[James Cownie]

Joachim Worringen wrote :-
> > No, you are confusing two completely different issues. To support
> > OpenMP you need a single address space which spans the processors.
> 
> You are right, this is completely different. However, I did not mean
> that connecting nodes of a cluster with a cache-coherent interface
> "gives you an SMP", but more precisely "gives the shared parts of the
> distributed distinct address spaces nearly SMP-like access
> characteristics", with respect to a suitable programming model. 

...

> With Quadrics, this should be possible in an even more efficient manner
> due to the hardware-MMU and -TLB on the adapter.

(One caveat, I'm assuming here that the Quadrics' model remains the
same as it was when we were all back at Meiko).

I think you still do not understand Quadrics' model. There is _no_
shared part of the address space. Access to remote address spaces is
never achieved directly by an arbitrary load/store from any CPU, it
always requires an access to the communication processor.

As I said before the model is of _explicit_ remote store access. You
have to generate different instructions to perform a remote access.

On the other hand, all remote accesses are fully cache coherent both
locally and remotely.

The issue of cache coherence of the interface is unrelated to the
issue of how you cause a remote store access.

> To have a real cc-NUMA-SMP, the integration needs to be higher (HP
> X-Class, DG/IBM NUMA-Q, ...), this is for sure.  The question is: are
> large-scale SMPs as sold by IBM, Sun, ... not the better solution for
> such tasks? Quadrics is expensive, and you still have to manage a bunch
> of PCs instead a nice, single SMP.

But, as I said, Quadrics' doesn't pretend to be a cc-NUMA-SMP at
all. Their technology is used to build _big_ clusters which may contain
the SMPs as nodes, but certainly scale above the range where the SMPs
run out. (See the recently announced HP/Quadrics PNL cluster, or the
Compaq SCs at LANL and CEA, for instance).

-- Jim 

James Cownie	<jcownie at etnus.com>
Etnus, LLC.     +44 117 9071438
http://www.etnus.com