[Beowulf] Digital Image Processing via HPC/Cluster/Beowulf - Basics
prentice.bisbal at rutgers.edu
Mon Nov 5 12:55:40 PST 2012
This article is from 14 years ago, but it might be relevant to your
situation. It describes how Digital Domain used a Linux 'render farm'
to do the GCI for Titanic. I haven't read this article in 14 years, so
I'm a little fuzzy on the details, but I think you might learn something
useful from it.
Manager of Information Technology
Rutgers Discovery Informatics Institute (RDI2)
On 11/03/2012 10:12 AM, CJ O'Reilly wrote:
> Thank you very much!
> I'll be sure to talk to the software developer about this.
> For now this project is moving slowly; still doing research (it's
> possible simply a single powerful computer could get this work done
> Perhaps I'll be back around in the future though!
> Thanks a bundle:)
> On Sat, Nov 3, 2012 at 9:50 PM, Lux, Jim (337C)
> <james.p.lux at jpl.nasa.gov <mailto:james.p.lux at jpl.nasa.gov>> wrote:
> 1. Yes and no.. The application process needs to be "parallel
> aware", but for some applications that could just mean running
> multiple instances, one on each node, and farming the work out
> to them. This is called "embarassingly parallel" (EP).. A good
> example would be rendering animation frames. Typically each
> frame doesn't depend on the frames around it so you can just
> parcel the work at a frame granularity to the nodes. There
> are other applications which are more tightly coupled and
> where the computation process running on node N needs to know
> something about what's running on Node N+1 and Node N-1 very
> frequently. For this, applications use some sort of
> standardized process communication library (e.g. MPI), or,
> perhaps a library that performs a high level function (e.g.
> Matrix inversion) that underneath uses the interprocess comm.
> 2. Another "it depends". If the process is EP, and each node is
> processing a different image, then your problem is one of sending
> and retrieving images, which isn't much different from a
> conventional file server kind of model. If multiple
> processors/nodes are working on the same image, then the
> interconnect might be more important. It all depends on the
> communication requirements. Note that even EP applications can
> get themselves fouled up in network traffic (imagine booting 1000
> nodes simultaneously, with them all wanting to fetch the boot
> image from one server simultaneously)
> This is the place to ask..
> From: CJ O'Reilly <supaiku at gmail.com <mailto:supaiku at gmail.com>>
> Date: Wednesday, October 31, 2012 11:31 PM
> To: "beowulf at beowulf.org <mailto:beowulf at beowulf.org>"
> <beowulf at beowulf.org <mailto:beowulf at beowulf.org>>
> Subject: [Beowulf] Digital Image Processing via
> HPC/Cluster/Beowulf - Basics
> Hello, I hope that this is a suitable place to ask this, if not, I
> would equally appreciate some advice on where to look in lue of
> answers to my questions:
> You may guess that I'm very new to this subject.
> I am currently researching the feasibility and process of
> establishing a relatively small HPC cluster to speed up the
> processing of large amounts of digital images.
> After looking at a few HPC computing software solutions listed on
> the Wikipedia comparison of cluster software page (
> http://en.wikipedia.org/wiki/Comparison_of_cluster_software ) I
> still have only a rough understanding of how the whole system works.
> I have a few questions:
> 1. Do programs you wish to use via HPC platforms need to be
> written to support HPC, and further, to support specific
> middleware using parallel programming or something like that?
> Can you run any program on top of the HPC cluster and have it's
> workload effectively distributed? --> How can this be done?
> 2. For something like digital image processing, where a huge
> amount of relatively large images (14MB each) are being processed,
> will network speed, or processing power be more of a limiting
> factor? Or would a gigabit network suffice?
> 3. For a relatively easy HPC platform what would you recommend?
> Again, I hope this is an ok place to ask such a question, if not
> please help refer me to a more suitable source.
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin Computing
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Beowulf