DBases in very large RAMDisks

[Robert G. Brown]

On Fri, 9 Mar 2001, Sergiusz Jarczyk wrote:

> Welcome
> This topic was discussed on many lists several times, and always ideas was
> crashed by one simple question - what happen when server crashes, or
> simply power goes down ? You can syncing data from memory with data on
> disks, but if you'll doing this in short period, overall performance won't
> be differ so much from "classical" implementations.

The same problem exists even if you leave the data on disk.  A lot of
disk I/O on Unixoid systems is buffered and cached in both directions
anyway by the operating system, and only is guaranteed to be written to
disk by e.g. the sync command or read from the disk if the file has been
altered or touched after its last read. If some particular part of the
DB is most commonly accessed, it will likely find its way into cache on
any sufficiently large-memory system and queries will be answered out of
cache instead of off the disk anyway.  I think that cache functions like
a FIFO and that pages persist until the space is recovered for other
purposes (e.g. mallocs or to run processes or to load active pages from
running processes), but I'm not absolutely certain.

One interesting question is therefore how much performance benefit you'd
actually gain by running from a ramdisk vs letting the OS cache the disk
(and running effectively from a ramdisk) but also letting the OS handle
items such as sync'ing the VFS with the actual file on disk.

You might do just as well by buying/building a system with 1-2 GB of
memory (lots of room for cache and buffers) and running only the DB
application plus the OS plus (perhaps) an application that at boot time
"reads" the entire DB as a file image, effectively preloading the disk
cache.  There might also be some way of tuning the OS to cache the file
pages more aggressively or to increase the size of its tables of pages
so that it can cache the entire DB at once, but I don't really know what
those limits are in linux and it may not be necessary.

   rgb

>
> Sergiusz
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>

-- 
Robert G. Brown	                       http://www.phy.duke.edu/~rgb/
Duke University Dept. of Physics, Box 90305
Durham, N.C. 27708-0305
Phone: 1-919-660-2567  Fax: 919-660-2525     email:rgb at phy.duke.edu