Tweaking the yellowfin

Jason Holmes jholmes@psu.edu
Fri Apr 23 13:37:44 1999


Tom Daley wrote:
> 
> > So far so good.  Everything seemed to be working.  I then ran PingPong
> > and found to my dismay that it topped out at 11 MB/s (the 100Mb cards
> > maxxed at 9 MB/S).  Running PingPing was fine until the packet size
> > reached 128 bytes.  Then it died and wouldn't come back.
> 
> You didn't mention what your hardware was.

Oops... sorry about that.  The compute nodes are single processor PII
266's with 256 MB RAM, everything SCSI, etc, etc.  The controlling node
is a dual 400 Mhz Xeon (1 MB L2 each) with 1 Gig of RAM (running at 960
MB).

> Try running your tests against the loopback (127.0.0.1) this will
> tell you how fast your machine can send packets if the adapter were
> perfect.
> 
> On a PentiumII 233 Mhz system I can get 23 MB/s.  This is considering
> that both sender and receiver are on the same box.  I guess the
> single ended performance would be around 40 MB/s with an adapter.

The PingPong test using the loopback interface on the controlling node
yields a maximum of around 35 MB/s, though things seem to be erratic. 
Here are some of the numbers:

       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000       758.98         0.00
            1         1000       127.37         0.01
            2         1000       903.11         0.00
            4         1000       585.79         0.01
            8         1000       526.34         0.01
           16         1000       133.44         0.11
           32         1000       790.88         0.04
           64         1000       126.54         0.48
          128         1000       134.33         0.91
          256         1000       130.12         1.88
          512         1000       919.68         0.53
         1024         1000       546.23         1.79
         2048         1000       363.50         5.37
         4096         1000      1272.40         3.07
         8192         1000       443.02        17.63
        16384         1000       613.48        25.47
        32768         1000       951.52        32.84
        65536          640      3631.66        17.21
       131072          320      3497.53        35.74
       262144          160     28394.81         8.80
       524288           80     17912.97        27.91
      1048576           40     39173.99        25.53
      2097152           20     83220.30        24.03
      4194304           10    165105.95        24.23

As a little update to my first email, I realized that when doing my
previous tests, I still had my NFS partitions mounted through the
100mbit NICs (we mount /home and /usr/local across the compute nodes
from the controlling node).  Mounting them through the gnics gave some
better numbers, though still topping out at around 15 Megs for the
PingPong test and 9 Megs for PingPing (results for the 100mbit NICs were
around 9 Megs and 6 megs, respectively.)  Here's the PingPong for the
gnics between two of the compute nodes:  

       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000       166.67         0.00
            1         1000       169.88         0.01
            2         1000       199.83         0.01
            4         1000       168.07         0.02
            8         1000       174.31         0.04
           16         1000       167.46         0.09
           32         1000       170.93         0.18
           64         1000       171.20         0.36
          128         1000       176.16         0.69
          256         1000       176.95         1.38
          512         1000       182.43         2.68
         1024         1000       229.68         4.25
         2048         1000       301.83         6.47
         4096         1000       442.76         8.82
         8192         1000       727.05        10.75
        16384         1000      1192.75        13.10
        32768         1000      2162.46        14.45
        65536          640      4111.61        15.20
       131072          320      8568.10        14.59
       262144          160     17876.03        13.99
       524288           80     37207.57        13.44
      1048576           40     75025.60        13.33
      2097152           20    154561.38        12.94
      4194304           10    307274.80        13.02

Has anyone else run the PingPong and PingPing MPI benchmarks with the
Yellowfin cards and the FDR?  Maybe these results are normal for this
test.  (PingPong and PingPing are part of the PALLAS MPI Benchmark
Suite.  The PALLAS version is 2.1 and our MPI implementation is mpich
1.1.2).

--
Jason Holmes


> I am currently waiting for some 400 Mhz PentiumII machines with
> 64 bit PCI before doing any more performance testing.
> 
> --
> -------------------------------------------------------------------
> |         o    Tom Daley                                          |
> |   ___ </v    Woodland Park, CO                                  |
> |  ___  -\     tdaley@crosstor.com                                |
> | ___    /                                    (719) 534-0449  x27 |
> |       (*)    Linux!                                             |
> -------------------------------------------------------------------
 | To unsubscribe, send mail to Majordomo@cesdis.gsfc.nasa.gov, and within the
 |  body of the mail, include only the text:
 |   unsubscribe this-list-name youraddress@wherever.org
 | You will be unsubscribed as speedily as possible.