[Beowulf] NVIDIA GPUs, CUDA, MD5, and "hobbyists"

[Vincent Diepeveen]

Prentice,

No one doubts your collegue.

Taking a md5 is very fast on PC processors. The code is very simple.  
Just a few lines of code it is.

At my raid10 array i take a lot of md5sums for big files (chess  
endgame table bases),
the limiting factor is the i/o speed.

I/O delivers oh in my case a 100MB/s.

Running 128 parallel sessions of md5sum is not so interesting at all,  
we all believe this can be done fast.

Even if you would have a harddrive that can deliver 20 GB/s, still  
the limiting factor taking md5sum with a GPU is
the PCI-e bandwidth of what is it say 2GB/s or  so?

So you can never stream enough data to the videocard to declare that  
it is useful to handle your md5sums,
if your  main processor, being a cheapo PC processor from years ago,  
already is doing more than fine there,
considering the bottleneck.

So this experiment of your high esteemed collegue is an example of an  
application that you do not want to buy those cards for.

Of course the reason he did do it, is obvious. He wanted to know  
something about the throughput it can deliver.

Yet the limitation of that throughput, namely that you have that  
throughput only within its registerfile/local cache,
means basically bad news.

Maybe it is an idea for your collegue to measure how many  
instructions per second he managed to get executed.
For the new cards getting close to say 0.15 Tera instructions per  
cycle would be interesting information to have.

Knowing the effective number of gigabytes per second it can process  
for md5 is however not interesting at all.

Because the above is so trivial, i guess that caused some here.

Interesting of course is to parallellize taking a md5. Yet being able  
to parallellize taking a md5sum  of course defeats the
original purpose of the algorithm, as it means of course by deduction  
of that fact that it is possible to fool md5sum (modifying
a file such that it contains the content you want it to have  
meanwhile giving the same md5sum output).

MD5 is hopelessly outdated for that reason, as it is very insecure,  
giving online hackers opportunities to do bad things at your
file system.

Vincent

On Jun 18, 2008, at 4:51 PM, Prentice Bisbal wrote:

> I seem to have muddied the waters of the original NVIDIA/CUDA post.
> Someone made the inaccurate statement that CUDA programming is  
> difficult
> and time-consuming. I cited the MD5-example of my colleague as an
> example of how easy it is to port the code, and how significant the
> performance improvements could be.
>
> Some subscribers to this list questions these results,and the  
> discussion
> quicky turned away from NVIDIA/CUDA/GPUs to MD5. I forward the e-mails
> to my colleague. Since he doesn't subscribe to this list, I'm replying
> based on information he has provided me. He hasn't asked me to  
> reply on
> his behalf, I'm doing this on my onw to contribute to the discussion.
>
> My colleague who did this work, Mario Juric, is a member at the
> Institute for Advanced Study (member = postdoc) studying Astrophysics.
> Contrary to the assertion by Vincent that CUDA/GPUs are only for
> hobbyists, Mario is very interested in using GPUs to speed up his
> astrophysics research. The biggest hindrance to doing "real" work with
> GPUs is the lack of dual-precision capabilities. As we all know, that
> hindrance was eliminated yesterday.
>
> In November of 2007, Mario organized the AstroGPU Workshop here at the
> Institute to discuss the use of GPUs in Astrophysics
> (http://www.astrogpu.org/). This workshop serves as proof that  
> there are
>  scientists serious about using GPUs for real work (not just  
> hobbyists).
>
> Now about that MD5 discussion... I forwarded Mario the replies to my
> post, and he replied thusly:
>
> <quote>
>
> Hi Prentice,
>     The guy ... is right -- it's throughput for a lot of simultaneous
> computations, not for a computation of a single hash. If you  
> attempt to
> compute a single hash on an entire card, you won't get any  
> improvement.
> Same as you wouldn't if you tried it on a single vs. quad core CPU.  
> But
> if you compute four hashes, than single vs. quad makes a huge
> difference. And the GPU cards are effectively 128 core CPUs, so  
> when you
> need to compute millions of hashes...
>
> Feel free to e-mail them my online writeup at:
>
>     http://majuric.org/software/cudamd5/
>
> The full source code is there as well. That should clear a lot of  
> things
> up.
>
> </quote>
>
> For those of you with questions about the MD5 performance, please see
> the link above, and better yet, take a look at Mario's code. The link
> includes lots of pretty graphs ;)
>
> <quote>
>
> Elcomsoft did the same thing, and sells it commercially:
>
> http://www.engadget.com/2007/10/24/elcomsoft-turns-your-pc-into-a- 
> password-cracking-supercomputer/
>
> </quote>
>
> NVIDIA has promised us some new GPUs through their Professor Partner
> program. I'm sure once we get our hands on them, we'll do more
> coding/benchmarking. Not sure if they'll be DP-capable units.
>
> -- 
> Prentice Bisbal
> Linux Software Support Specialist/System Administrator
> School of Natural Sciences
> Institute for Advanced Study
> Princeton, NJ
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit  
> http://www.beowulf.org/mailman/listinfo/beowulf
>