[Beowulf] Pretty Big Data

Perry E. Metzger perry at piermont.com
Mon Jan 25 08:21:02 PST 2016


On Mon, 25 Jan 2016 15:06:44 +0000 "Lux, Jim (337C)"
<james.p.lux at jpl.nasa.gov> wrote:
> Figure that RAM has a read cycle time of 1 ns (DDR3 or DDR4)

But not a *latency* of 1ns.

> High speed disk drives (in the spinning rust, sense) in ³commodity²
> kinds of speeds seem to be in the 7200 RPM range, which is a 4
> millisecond average latency. 

But everyone uses SSD at this point for any real apps, and generally
directly attached to the PCIe.

> So, if we¹re comparing searching a TB of data in RAM vs searching a
> TB of data on a disk, I think the RAM is always going to win if
> it¹s a sequential search.

Sure, but our original question was a sequential search of RAM vs. an
indexed search of mass storage.

The point I was making was that no amount of RAM will protect you
from sufficiently stupid data structure choices if the data is
big enough and you hit the data structures hard and often enough.

Perry
-- 
Perry E. Metzger		perry at piermont.com


More information about the Beowulf mailing list