[Beowulf] posting bonnie++ stats from our cluster: any comments about my I/O performance stats?

[Joe Landman]

Rahul Nabar wrote:
> On Fri, Sep 25, 2009 at 2:38 PM, Rahul Nabar <rpnabar at gmail.com> wrote:
> 
>>> We use this to model bonnie++ and other types of workloads.  It provides a
>>> great deal of useful information.
>>
> 
> More details from the fio benchmark.....
> 
> ##################################################################################################
> fio: terminating on signal 2
> 
> random: (groupid=0, jobs=16): err= 0: pid=26509
>   read : io=12,391MiB, bw=3,732KiB/s, iops=466, runt=3399938msec
>     clat (msec): min=3, max=1,122, avg=23.06, stdev= 4.38
>     bw (KiB/s) : min=    7, max=  621, per=9.34%, avg=348.67, stdev=29.11
>   cpu          : usr=0.04%, sys=0.17%, ctx=3736755, majf=0, minf=33036
>   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
>      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
>      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
>      issued r/w: total=1586090/0, short=0/0
> 
>      lat (msec): 4=0.01%, 10=0.38%, 20=17.11%, 50=69.74%, 100=12.01%
>      lat (msec): 250=0.73%, 500=0.02%, 750=0.01%, 1000=0.01%, 2000=0.01%

Looks like your IOP latency is around 50ms.  If you think about this, it 
  seems a little high, even for a RAID5.  I'll do some measurements here 
on our big units and we can compare.

466 IOP is ok ... basically 4 drives will give you this (5 RAID5 drives 
-> 4 drives of data).  If you are IOP bound, you can do better, if this 
matters.


-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615