<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head><title></title>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta http-equiv="Content-Style-Type" content="text/css">
<style type="text/css"><!--
body {
margin: 5px 5px 5px 5px;
background-color: #ffffff;
}
/* ========== Text Styles ========== */
hr { color: #000000}
body, table /* Normal text */
{
font-size: 9pt;
font-family: 'Courier New';
font-style: normal;
font-weight: normal;
color: #000000;
text-decoration: none;
}
span.rvts1 /* Heading */
{
font-size: 10pt;
font-family: 'Arial';
font-weight: bold;
color: #0000ff;
}
span.rvts2 /* Subheading */
{
font-size: 10pt;
font-family: 'Arial';
font-weight: bold;
color: #000080;
}
span.rvts3 /* Keywords */
{
font-size: 10pt;
font-family: 'Arial';
font-style: italic;
color: #800000;
}
a.rvts4, span.rvts4 /* Jump 1 */
{
font-size: 10pt;
font-family: 'Arial';
color: #008000;
text-decoration: underline;
}
a.rvts5, span.rvts5 /* Jump 2 */
{
font-size: 10pt;
font-family: 'Arial';
color: #008000;
text-decoration: underline;
}
span.rvts6
{
font-weight: bold;
color: #800000;
}
span.rvts7
{
font-weight: bold;
color: #800080;
}
/* ========== Para Styles ========== */
p,ul,ol /* Paragraph Style */
{
text-align: left;
text-indent: 0px;
padding: 0px 0px 0px 0px;
margin: 0px 0px 0px 0px;
}
.rvps1 /* Centered */
{
text-align: center;
}
--></style>
</head>
<body>
<p>Hallo Joe,</p>
<p><br></p>
<p>Donnerstag, 28. Februar 2008, meintest Du:</p>
<p><br></p>
<p><br></p>
<p><span class=rvts7>>> The best i saw for NFS over 10 GE was about 350-400 MB/s write and about 450 MB/s read.</span></p>
<p><span class=rvts7>>> Single server to 8 simultaneous accessing clients (aggregated performance). </span></p>
<p><br></p>
<p>The clients had a 1 gig uplink... </p>
<p><br></p>
<p><span class=rvts6>JL> Hi Jan:</span></p>
<p><br></p>
<p><span class=rvts6>JL> Ok. Thanks. This is quite helpful.</span></p>
<p><br></p>
<p><br></p>
<p><span class=rvts7>>> On the blockdevice i got 550 MB/s write and 1.1 GB/s read performance. </span></p>
<p><br></p>
<p><span class=rvts6>JL> Using iSCSI? To real disks or ramdisk/nullio? Most of the benchmarks I</span></p>
<p><span class=rvts6>JL> have seen online have been to nullio or ramdisks. We are going to real</span></p>
<p><span class=rvts6>JL> disks.</span></p>
<p><br></p>
<p>Real disks. 16 SAS 15k Disks on a LSI 8888 Controller in RAID-5. Connected through a x4 SAS connection on a backplane. Because of the x4 SAS connection the read rate is limited to 1.1 gig/s - with discrete connections to the drives the read speed should be 50% higher. </p>
<p><br></p>
<p>I couldn't get a tmpfs exported over NFS - but i did not try very hard on that because it does not make any sense for practical usage - just to find out if NFS itself is the bottleneck.</p>
<p><br></p>
<p>I tried several configs including software raid-0. The performance was a disaster compared to theoretical values. </p>
<p><br></p>
<p><span class=rvts7>>> JL> 2.6.23.14 kernel on both sides, jumbo frames enabled. No switch, just</span></p>
<p><span class=rvts7>>> JL> a CX4 cable.</span></p>
<p><br></p>
<p><span class=rvts7>>> rsize/wsize are set to? </span></p>
<p><br></p>
<p><span class=rvts6>JL> I tried a range: 8k through 64k</span></p>
<p><br></p>
<p>Okay. That was the most important improvement i could do. The first kernel i used did not allow to go over 8k - with 32k if was much faster. </p>
<p><br></p>
<p><span class=rvts7>>> NFS3 or NFS4? </span></p>
<p><br></p>
<p><span class=rvts6>JL> 3.</span></p>
<p><br></p>
<p>I saw no performance improve with NFS4 i have to say. Everybody i talked to pointed at the bad NFS performance of linux (and many said: use solaris - it is much faster ;-) )</p>
<p><br></p>
<p>Cheers,</p>
<p>Jan</p>
</body></html>