[Beowulf] Naming Convention Survey

[David Kewley]

On Monday 09 January 2006 11:28, Brian D. Ropers-Huilman wrote:
> Simple question, likely a complicated answer. Cluster systems should
> have up to three distinct file systems:
>
> 1) a place to compile and keep "stuff"
> 2) a bit of disk to use while code is running
> 3) potential a bit of disk with special characteristics, likely
> high-performance
>
> Typically #1 would be called /home. Here at LSU, #2 is typically
> called /var/scratch or /var/local/scratch. Then there is #3.
> Sometimes we call this /scratch and sometimes /work.
>
> My question: what do other folks call these bits of disk?

Hmm.  We have a central specialized filesysteme (35TB Ibrix), plus about 
23GB free hard disk on each compute node.  /home is on the Ibrix 
filesystem.  /scratch is on the node's local hard disk.

In principle (I have yet to convince myself that this holds in practice, 
given our current conditions), a user will use /home if they want 
storage that has some of these characteristics:

* low latency (faster than local disk, it's claimed)
* high bandwidth (up to about 1GB for I/O to many large files)
* visible to all nodes
* large capacity
* intended to have more than transient use

In general, we suggest people use /home for most uses.  But node-local 
transient storage should go on /scratch.  And if a lot of nodes are 
trying to do I/O to the central filesystem all at once (many 
randomly-distributed files), at a certain attempted bandwidth or 
#files/sec, the central storage will saturate, and it may be better to 
write to local disk and collect it later.

I haven't thought out all these usage cases fully, but you get the idea.  
It bears some thought in each usage case.

David