[Beowulf] Striped file system with RAM disk.

[Alan Louis Scheinine]

    There is a particular kind of application, single-client
and serial process, for which a striped file system using
RAM disk would be very useful.  Consider reading small
blocks at random locations on a hard disk.  The latency
of the HDD could be a few milliseconds.  Adding more HDD's
does not solve the problem, unlike an application based on
streaming.  Adding more disks and parallelizing the program
could be a solution but sometimes there is no time
to parallelize the program.

    A possible solution is RAM disk.  But if we put, for example,
64 GB of RAM on a single computer then that computer becomes
specialized and expensive, whereas the need for a huge
amount of RAM may be only temporary.  An alternative is to
use a cluster of nodes, a typical Beowulf cluster.  For example,
using a striped file system over 16 nodes where each node has 4 GB
of RAM.  Each node would have a normal amount of RAM and yet
could provide the aggregate storage of 64 GB when the need arises.
While we have not yet created this configuration, I suppose
that Gbit Ethernet could provide 100 microsecond latency and
Infiniband or Myrinet could provide 10 microsecond latency.
Much, much less than the seek time of a HDD.

    The idea is so simple that I imagine it has already been done.
I would be interested in learning from other sites that have
used this method with a file system such as Lustre, PVFS2 or
another.

best regards,
Alan Scheinine

-- 

  Centro di Ricerca, Sviluppo e Studi Superiori in Sardegna
  Center for Advanced Studies, Research, and Development in Sardinia

  Postal Address:               |  Physical Address for FedEx, UPS, DHL:
  ---------------               |  -------------------------------------
  Alan Scheinine                |  Alan Scheinine
  c/o CRS4                      |  c/o CRS4
  C.P. n. 25                    |  Loc. Pixina Manna Edificio 1
  09010 Pula (Cagliari), Italy  |  09010 Pula (Cagliari), Italy

  Email: scheinin at crs4.it

  Phone: 070 9250 238  [+39 070 9250 238]
  Fax:   070 9250 216 or 220  [+39 070 9250 216 or +39 070 9250 220]
  Operator at reception: 070 9250 1  [+39 070 9250 1]
  Mobile phone: 347 7990472  [+39 347 7990472]