[Beowulf] noob understanding

[Robert G. Brown]

On Thu, 18 May 2006, phildlight at verizon.net wrote:

> I've searched many of the posts and I only have a few quick questions.
> Please excuse my ignorance.
>
> 1. Is my understanding that you can process windows processes on a
> linux cluster correct?
>
> 2. Specifically, I'm interested in using a linux cluster (25 nodes)
> for rendering 3d studio max by splitting single frames accross the
> cluster.  Example.  We work with enormous output files, usually around
> 5400x3600, and they render in about 15 hours on a dual xeon 3.4ghz w/
> 2gb ram.  Distributed rendering via windows works, but we're interested
> in growing the "farm" to many more nodes, hence the interest in a
> beowulf.  Occasionally, we'll do some animations also, so it again would
> be enormously beneficial.  I think I understand that this is possible,
> and am interested more specifically in the scripting method to dump the
> job into the cluster from windows.

In principle this is possible, but the key is the rendering software and
how you distrbute the jobs.  You cannot "process windows processes" on a
linux cluster as Windows applications are binary incompatible with
linux.  Exceptions to this exist (using e.g. windows emulation
libraries) but are unlikely to give you much of a performance boost --
more likely the opposite.

If you can find linux-based software for doing the actual rendering from
the scene description language then there are at least two ways to
proceed -- with a parallelized version of the rendering program (e.g.
POVray-PVM) or by putting a single frame on each node as you are doing
now.  Both should probably be investigated because of the possibility of
superlinear speedup -- if you are memory bound in any of several
possible ways there may be some point in partitioning the images rather
than running them embarrassingly parallel.

There are several ways of getting the jobs onto the cluster from
Windows, as well, although if you have the software that runs under
Linux in the first place, what's the point?  If the issue is just
distributing a stack of frame descriptors and getting them turned into
images, you could do this lots of ways from any sort of host (for
example, writing a PVM master program on the Windows side that sends
frames to slave rendering programs running on the Linux side.

Beyond that, I'm not enough of an expert to know what software is out
there for pro-grade rendering under linux.  GIYF, of course -- for
example, it turned up this:

   http://linux.omnipotent.net/article.php?article_id=8043

Of course Pixar uses Renderman, which runs on Maya, on Linux -- Shrek
was rendered using Linux.  See e.g.:

   http://cgw.pennnet.com/Articles/Article_Display.cfm?Section=Archives&Subsection=Display&ARTICLE_ID=118664

or numerous other hits from Google.  So it may or may not be possible
with "3d studio max" but it definitely is possible...;-)

> Additionally, we use vray (render engine plugin for max) for the final
> output.
>
> Am I correct in thinking that this is possible?  Any direction at all
> would be great!  Thanks.

As I said, Google Is Your Friend here.  the 3d studio max folks should
also be able to help you -- if they have a linux version (or know of a
linux version) for the actual frame rendering portion they should be
able to direct you to it.

I think you're smart looking into this as I would bet that the linux
versions of the renderers run faster than the windows versions on the
otherwise identical hardware because of better memory management if
nothing else.  Also consider using AMD Opterons instead of Xeons -- I'd
bet money that they will be considerably faster for this sort of thing.
You may have to invest some money in prototyping the process, but in the
long run it should pay off handsomely.

   Good luck!

      rgb

>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu