[Beowulf] OS for 64 bit AMD

[Joe Landman]

Hi David:

David Kewley wrote:
> Joe Landman wrote on Thursday 31 March 2005 10:04:
>>Actually Redhat now has HPC pricing per node.  There are other good
>>reasons to look elsewhere for HPC distributions though, specifically due
>>to their lack of good high performance/scalable per-node file system.
> 
> In your experience/knowledge, what are the issues that make e.g. xfs better 
> than ext3 on RHEL4?  I ask specifically about RHEL4 because RH has worked 
> hard on ext3 in time for RHEL4.

RHEL4 is too new to talk about conclusively.  Oddly enough, the same 
language was used for RHEL3, that Redhat has worked really hard on 
improving ext3.

In real world applications (large data set analysis codes, and related) 
it is reasonably well known that you get better performance out of ext3 
by turning off journaling, effectively making it ext2.  Ext3 has a 
number of serialization and blocking issues associate with its 
journaliing code, not to mention other issues (directories with many 
files comes to mind).

The I/O dominated/intensive applications that I have seen or worked with 
using xfs vs ext3 have demonstrated significant performance advantages 
to xfs.  Due to NDA's I cannot talk about specific details though the 
performance differences were significant.

> My understanding is that RH choose to support ext3 but not xfs because: 1) 
> they have in-house expertise for ext3 but not for xfs, and 2) they believe 
> that xfs has no real advantages over ext3.

They have many customers that disagree with 2, and 1 is a tautology, as 
they have invested heavily in ext3, and thus they have people to work on 
it.

Some inherent disadvantages of ext3 show up when you start looking at 
large file systems and large files.  Xfs has much higher limits.  If you 
want to build a 30TB file system across a huge disk array attached to a 
sizeable SMP machine, can you do it with ext3?  (no as of RHEL3).  If 
you want to work with a 2.5 TB file (part of a recent benchmark we ran), 
can you do it with ext3?  (no as of RHEL3).  Xfs doesn't have a problem 
with either of these.

> If customers show RH that there are real-life needs for xfs that are not 
> satisfied by ext3, then RH may well be willing to invest in in-house xfs 
> expertise.

Unlikely.  Customers have been showing a clear need for this for a while 
(Sloan sky survey, and many others with huge and high speed data 
requirements).  Redhat prefers to use the excuse that it is a large and 
complex package.  Hmmm.  So are Xorg, Openoffice, ....

I do not expect Redhat to do this.  SuSE has, as have most of the rest 
of the major distributions (including the 1 man distribution shops), so 
the excuses that one hears are ... well ... probably not the real 
reasons.  Redhat does not want to promote a competitor to technology it 
supports.  That seems to be a simpler explanation, and I believe is 
better supported by observing their actions.

Note:  FC-x has xfs enabled, one only needs to use "linux xfs" on the 
boot line during installation.

> 
> David
> _______________________________________________
> Beowulf mailing list, Beowulf at beowulf.org
> To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics LLC,
email: landman at scalableinformatics.com
web  : http://www.scalableinformatics.com
phone: +1 734 786 8423
fax  : +1 734 786 8452
cell : +1 734 612 4615