[Beowulf] Hadoop's Uncomfortable Fit in HPC

Jason Riedy jason at lovesgoodfood.com
Tue May 20 07:34:50 PDT 2014


And Ellis H. Wilson, III writes:
> 4. Although I'm not the biggest fan of HDFS, but this post misses the
> /entire/ point of HDFS: reliability in the face of (numerous)
> failures.

This is what I keep reminding people when they rag on HDFS
performance.  Most of HPC tries adding just enough fault
tolerance without overly hindering performance (and pretty much
fails outside c/r).  HDFS tries to add performance without
hindering fault tolerance.  Different design space, and quite
worth watching.

(Although the current inane (yes, inane) practice of placing MR
jobs on nodes separate from the storage baffles me.)




More information about the Beowulf mailing list