[Beowulf] Compute Node OS on Local Disk vs. Ram Disk

[Douglas Eadline]

Good point about swap. I often try an make the distinction
that diskless booting (provisioning) does not require
diskless nodes. That is, it is perfectly reasonable to
use a centralized provisioning method and yet have HDD's
on compute nodes -- if you need them. In the case where
swap and local scratch space are needed this makes sense.
(As long as the disks don't start collecting
any cluster state on them)

--
Doug


> Hi,
>
> Am 29.09.2008 um 12:27 schrieb Alan Ward:
>
>> Ram disks, definately. ;-)
>>
>> Afraid I'm still going with diskless nodes. You save
>>
>> 1. some money on the disks themselves
>> 2. more money on solving disk failures
>>
> this highly depends on the used applications. Quantum chemistry code
> like Gaussian03 or Molcas would run much slower, when all  scratch
> files would be written to NFS instead of a local disk. We even use
> two or three stripped disks in the nodes.
>
> As the disk is in the node because of our applications anyway, I also
> put the OS there.
>
> -- Reuti
>
>>
>> 3. yet more money on cooling
>>
>> This may be specially important in a high-density rack situation,
>> where if you get the disks out of the way each box can get more
>> ventilators up front as well as in the rear.
>>
>> On the other hand, Windows Vista has had its uses(!), such as
>> driving RAM prices down as demand expands ...
>>
>> Cheers,
>> -Alan
>>
>>
>>
>> -----Original Message-----
>> From: beowulf-bounces at beowulf.org on behalf of Jon Forrest
>> Sent: Mon 9/29/2008 6:44 AM
>> To: Beowulf Mailing List
>> Subject: [Beowulf] Compute Node OS on Local Disk vs. Ram Disk
>>
>> There are two philosophies on where a compute node's
>> OS and basic utilities should be located:
>>
>> 1) On a local harddrive
>>
>> 2) On a RAM disk
>>
>> I'd like to start a discussion on the positives
>> and negatives of each approach. I'll throw out
>> a few.
>>
>> Both approaches require that a compute node "distribution"
>> be maintained on the frontend machine. In both cases
>> it's important to remember to make any changes to this
>> distribution rather than just using "pdsh" or "tentakel"
>> to dynamically modify a compute node. This is so that the
>> next time the compute node boots, it gets the uptodate
>> distribution. Although the mechanism for maintaining
>> the distribution varies in either approach, I consider
>> this a push since one mechanism isn't inherently better
>> than the other.
>>
>> Assuming the actual OS image is the same in both cases,
>> #2 clearly requires more memory than #1. There are actually
>> two approaches to #2 - a) where only the OS and other stuff
>> necessary to boot the system are kept in memory and everything
>> else is in an NFS-mounted file system, and b) where the whole OS
>> installation is kept in memory. Depending on which approach
>> is taken, the RAM-based installations can take hundreds of MB
>> more than a local harddrive installation. However, on a modern
>> multicore compute node this might just be a few percent of the
>> total RAM on the node.
>>
>> Long ago not installing a local harddrive saved a considerable
>> about of money but this isn't true anymore. Systems that need
>> to page (or swap) will require a harddrive anyway since paging
>> over the network isn't fast enough so very few compute nodes
>> will be running diskless.
>>
>> Approach #2 requires much less time when a node is installed,
>> and a little less time when a node is booted.
>>
>> What are some of your favorite issues, positive or negative, with
>> each approach?
>>
>> Cordially,
>> --
>> Jon Forrest
>> Research Computing Support
>> College of Chemistry
>> 173 Tan Hall
>> University of California Berkeley
>> Berkeley, CA
>> 94720-1460
>> 510-643-1032
>> jlforrest at berkeley.edu
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit
> http://www.beowulf.org/mailman/listinfo/beowulf
>


--
Doug