[Beowulf] scaling / file serving

[Kumaran Rajaram]

Patrice,

   Having single NFS file server to cater to the storage demands of
cluster (50 or more compute nodes) may not be good solution. The
bottleneck is network bandwidth between the NFS storage node and rest of
the nodes. Probably you can add multiple GigE cards and configure the switch
with link aggregation to improve network bandwidth. Or go for high-speed
interconnection network (Infiniband). But still the max NFS I/O bandwidth
is limited by network. Well it also depends on the nature of your
application. If the compute nodes sparsely communicates with the NFS
server or communicate at different non-overlapping time interval  or some
staging in performed in compute node's local FS, you may not experience
this bottleneck. The bottleneck is evident when all the compute nodes pound the
NFS server at the same time. This may be reflected by the overall
execution time of the application across the cluster.

   Its better to dedicate multiple storage servers and install cluster
file systems like Sistina GFS, Lustre, Panasas, PVFS2, or Polyserve
to ensure global namespace. The local disk in the storage servers or SAN
can be used as storage device. Availability, Scalability, and
Storage Management is taken care/provided by the cluster file systems.
Hope this helps.

-Kums

__
Kumaran Rajaram
Verari Systems, Inc.
Phone: 205-314-3471 x208

On Wed, 9 Jun 2004, Patrice Seyed wrote:

> Hi,
>
> A current cluster with 15 nodes/ 30 processors mainly used for batch
> computing has one head/management node that maintains scheduling services as
> well as home directories for users. The cluster is due for an upgrade that
> will increase the number of compute nodes to about 100. I'm considering
> breaking out one of the compute nodes, adding disks, and making it a
> storage/file server node.
>
> At what point in scaling up would you folks recommend adding a node
> dedicated to file serving of home dirs, etc.? I have a separate cluster with
> 134 nodes purchased with the storage node in mind.
>
> Regards,
>
> Patrice Seyed
> Linux System Administrator - SIG
> RHCE, SCSA
> Boston University School of Medicine
>
>
>
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
>