[Beowulf] SSD caching for parallel filesystems
diep at xs4all.nl
Mon Feb 11 16:32:15 PST 2013
On Feb 12, 2013, at 1:13 AM, Lux, Jim (337C) wrote:
> Not simulations..
> Streaming the data out to a Digital to Analog converter..
> Basic box works like this:
> Attitude and guidance simulator gives a parallel bit word that
> indicates range and doppler and target type.. that's used to set a
> timer and frequency offset
> Radar gives a sync pulse to the target simulator that indicates
> that the pulse will be transmitted in a short time, and a parallel
> word telling it which kind of pulse it's going to transmit.
> Target simulator points to the correct pulse in the bulk storage
> which holds a bunch of simulated returns based on target type and
> pulse type (e.g. a 4 ns pulse will have a different return than a
> 1ms long pulse)
> Bulk store starts streaming the pulse into a FIFO. A countdown
> timer for range delay triggers the output of the FIFO to a DAC
> which generates the actual pulse.
> So you don't have to simulate the returns in real time.. .they're
> done off line in non-real time, and all the target box does is play
> back the correct return at the correct time and Doppler.
> You could use a lot less sample returns and just adjust the
> amplitude and phase, however, you'd like to simulate things like a
> lateral translation across the edge of a rocky area to a flat sandy
> area: the former has a much more ragged return that lasts longer,
> while the latter is a more specular reflection and narrower.
> If the beam is significantly tilted relative to the surface normal,
> the return is much longer in time than if it's pointing straight
> down, because the "spot" that is illuminated has significant range
> extent in the former case.
> This is why you wind up needing lots of different potential returns
> for the same length transmitted pulse.
> And that's for non-coded pulses. Throw in coding on the radar
> pulse (chirps or PN codes) and it gets even more interesting.
> One might even take a sample radar out to a test site in the desert
> (which looks an awful lot like the surface of Mars, at least to a
> radar) and record actual returns from different types of terrain.
> I don't recall the costs, but in any case, a more important design
> aspect is getting it to work at all, rather than driving the price
> down to the minimum.
> I was responding to your question asking for an example of
> something needing high bandwidth and small storage, but that
> couldn't be adequately addressed by just buying a ton of
> conventional RAM.
> These systems don't have a CPU.. it's just disk drive, FIFO, data
You're speaking here of a very specific NASA type problem.
It's like asking why one would design shoes for the guy who manages
to jump 10 meters high - and then you show up with an astronaut 10+
away from now who might jump on the Moon 10 meters high :)
> I suspect digital video recorders are another application.. very
> similar kind of usage..
> You stream raw video in at some Gbytes/second and just dump it to
> the drive(s). digital motion picture projectors at 4K resolution
> are probably another example. Although there, you are looking at
> fairly large data sets.
Not at all, you just want petabytes of storage there.
Actually the solutions sold until recently used a DEC alpha 1Ghz
21264 (i say from head), to steer the raid arrays.
Wasn't that around a 1GB/s bandwidth or so?
Price per petabyte of storage is everything there.
> The RED cameras stream to flash, SSD or conventional drives, for
> instance. 9Megapixels/frame*30 fps is 270 megapixels/second.
> The newer EPIC cameras do 31.8Mpix/frame * 96 fps... I think
> they run about 400-500 Mbyte/sec to a SSD "magazine"
> The film business is used to interchanging magazines. A typical
> 35mm cine camera has 400 ft and 1000ft magazines. At the usual
> 24fps, that works out to about 1 foot/second, so 400 or 1000
> seconds of shooting time. A comparable RED/EPIC magazine, then,
> probably holds half a terabyte or so.
> Here you go.. a 4 pack of 256GB drives for $9100...
This is embedded hardware, again not some HPC type workload.
Not a good example therefore.
> 256GB is probably comparable to a 400 foot magazine at the highest
> Jim Lux
> -----Original Message-----
> From: beowulf-bounces at beowulf.org [mailto:beowulf-
> bounces at beowulf.org] On Behalf Of Vincent Diepeveen
> Sent: Monday, February 11, 2013 3:30 PM
> To: beowulf at beowulf.org List
> Subject: Re: [Beowulf] SSD caching for parallel filesystems
> Ah you're doing the simulations a lot slower :) No big deal. You
> get what you pay for :)
> You're speaking about 1 box here or so?
> What size SSD array size is in that box, what bandwidth does it
> deliver to your CPU's and what price did you buy it for Jim?
> Then we can compare it with a harddrive raid array :)
More information about the Beowulf