[Beowulf] Parallel memory

[Randy Wright]

At cluster 2005 in Boston last month, I listened to a prsentation about
swapping over infinband's rDMA to remote memory. They had a latge
quicksort running at 1.7 times the speed of doing it in local memory only.

	http://www.cluster2005.org/122.html

--randy


On Tue, 18 Oct 2005, Robert G. Brown wrote:

> On Tue, 18 Oct 2005, Henderson, T Todd @ IS wrote:
> 
> > Are there any drivers, tools, etc. that can make the memory space on a
> > cluster look shared, something like pvfs but for memory?  I'm sure there
> > would be a speed hit, but in this instance, speed isn't the problem as much
> > as memory.  We have a code that uses a huge amount of memory and the memory
> > usage is proportional to the cube of the problem size, but the time for it
> > to run isn't too much of an issue.  I've been asked to parallelize the code
> > using mpi which is going to be a major effort.  However, I thought that if
> > there was anyway, even if inefficient speed wise, to create a virtual
> > parallel memory system it would be better than it using swap space and save
> > a bunch of coding time.
> 
> There have been projects like this in the past; I don't know what their
> current status is.  The Trapeze project at Duke (Jeff Chase) is pretty
> much exactly this -- creating a large VM (something like a CC-NUMA if I
> recall) using cluster nodes and myrinet for their MEMORY (and myrinet's
> relatively low latency for "memory-like" retrieval).  I think that the
> project is no longer active, but chase at cs.duke.edu (or maybe his student
> Justing Moore who frequents this list) can update you on it.
> 
> There was once also an online list discussion about swapping to an NFS
> mounted remote exported ramdisk, IIRC.  In principle all of that should
> work although I have NOT tried it and cannot say for certain.  Making a
> big ramdisk is fairly straightforward.  Once made and mounted, it can be
> NFS exported.  The trick then is whether it indeed possible to swap to
> an NFS mounted swapspace.  On Suns this used to be routine (otherwise
> e.g. the SLC and ELC, fully diskless sparc nodes from the 90's, wouldn't
> have worked).  On linux back in the 2.0 and maybe 2.2 series, I think
> that there was a problem -- maybe with page sizes? -- and it wouldn't
> work.  But I THINK that I recall hearing that remote linux swap now
> works, and if so that might be a way to go to get to a largish VM.
> 
> I'd really be interested (I'm sure the whole list would be) if you try
> the latter and it works, especially if it works pretty "well" (e.g.
> orders of magnitude faster than disk swap if not as fast as real
> local memory).  It would also give a great idea to all those cs grad
> students looking for a useful project, as very large VM systems are
> almost as advantageous to certain kinds of research as very FAST local
> memory systems are to others.
> 
> > Also, are there any tools to help implement mpi in an older code?
> 
> I've decided that I'm too ignorant to help on MPI questions, so I'll
> leave this to maybe Jeff Squyers and other people who are real experts;-)
> 
>     rgb
> 
> >
> >
> >
> > Any thoughts?
> >
> >
> >
> > Thanks,
> >
> > Todd
> >
> >
> >
> >
> 
> 

-- 

Randy Wright
Senior Systems Administrator
ITS/eServices
New York University
212-998-3164