Intial Pallas performance with Myrinet on a 860 & E7500

alex at compusys.co.uk alex at compusys.co.uk
Thu Apr 18 01:58:16 PDT 2002


For your information, please look at the following performance
measurements for the 'C' class Myrinet2000 cards.

Details of the two machines (optimisation level: -fast & PGI):

- 2.4.17 kernel
- mpich-1.2.1..7b
- gm-1.5.1
- measurement performed between machines

860 Supermicro DCE:
- Dual P4 2 GHz
- C class Myrinet2000

The new E75000 Supermicro DDR : 
- Dual P4 1.8GHz 
- C class Myrinet2000, using PCI-X slot


Notice the results for E75000 Sendrecv (and Exchange):  

4194304 --> 290.29Mbytes/s

That is more than the serverworks LE chipset.


Alex

(shown results are limited due to mailing list size limit)
///////////////////////// E75000 //////////////////////////////////////

#---------------------------------------------------
# Benchmarking PingPong 
# ( #processes = 2 ) 
#---------------------------------------------------
       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000         8.79         0.00
            1         1000         8.96         0.11
            2         1000         8.94         0.21
            4         1000         8.96         0.43
            8         1000         8.96         0.85
           16         1000         9.03         1.69
           32         1000         9.32         3.27
           64         1000         9.44         6.47
          128         1000        12.14        10.06
      2097152           20      8726.80       229.18
      4194304           10     17300.95       231.20

#---------------------------------------------------
# Benchmarking PingPing 
# ( #processes = 2 ) 
#---------------------------------------------------
       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000        11.97         0.00
            1         1000        12.64         0.08
            2         1000        12.09         0.16
            4         1000        12.87         0.30
            8         1000        12.33         0.62
           16         1000        12.36         1.23
           32         1000        11.73         2.60
           64         1000        11.90         5.13
          128         1000        14.65         8.33
      2097152           20     13792.25       145.01
      4194304           10     27535.80       145.27

#-----------------------------------------------------------------------------
# Benchmarking Sendrecv 
# ( #processes = 2 ) 
#-----------------------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]   Mbytes/sec
            0         1000        12.60        12.60        12.60         0.00
            1         1000        12.68        12.69        12.68         0.15
            2         1000        12.76        12.76        12.76         0.30
            4         1000        12.73        12.73        12.73         0.60
            8         1000        12.52        12.53        12.53         1.22
           16         1000        12.59        12.59        12.59         2.42
           32         1000        11.74        11.74        11.74         5.20
           64         1000        11.81        11.81        11.81        10.34
          128         1000        14.41        14.42        14.42        16.93
      2097152           20     13778.64     13778.80     13778.72       290.30
      4194304           10     27558.40     27558.70     27558.55       290.29

#-----------------------------------------------------------------------------
# Benchmarking Exchange 
# ( #processes = 2 ) 
#-----------------------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]   Mbytes/sec
            0         1000        20.93        20.93        20.93         0.00
            1         1000        20.96        20.97        20.97         0.18
            2         1000        20.96        20.97        20.96         0.36
            4         1000        20.96        20.97        20.97         0.73
            8         1000        21.03        21.05        21.04         1.45
           16         1000        21.01        21.02        21.02         2.90
           32         1000        21.30        21.30        21.30         5.73
           64         1000        21.33        21.34        21.33        11.44
          128         1000        23.98        23.98        23.98        20.36
      2097152           20     27563.40     27563.60     27563.50       290.24
      4194304           10     55052.30     55053.10     55052.70       290.63

#----------------------------------------------------------------
# Benchmarking Allreduce 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.15         0.15         0.15
            4         1000        19.22        19.23        19.23
            8         1000        19.25        19.26        19.25
           16         1000        19.31        19.33        19.32
           32         1000        20.02        20.03        20.03
           64         1000        20.39        20.40        20.40
          128         1000        25.96        25.97        25.96
      2097152           20     30421.15     30422.15     30421.65
      4194304           10     67887.70     67889.70     67888.70

#----------------------------------------------------------------
# Benchmarking Reduce 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.08         0.08         0.08
            4         1000        10.01        10.02        10.01
            8         1000        10.01        10.02        10.01
           16         1000        10.06        10.07        10.07
           32         1000        10.40        10.41        10.40
           64         1000        10.62        10.63        10.63
          128         1000        14.25        14.26        14.25
      2097152           20     25153.70     25231.65     25192.67
      4194304           10     49852.60     50856.40     50354.50

#----------------------------------------------------------------
# Benchmarking Reduce_scatter 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.53         0.55         0.54
            4         1000        21.00        21.00        21.00
            8         1000        21.39        21.40        21.39
           16         1000        21.29        21.30        21.29
           32         1000        21.70        21.71        21.70
           64         1000        22.37        22.38        22.37
          128         1000        25.39        25.40        25.40
      2097152           20     41247.20     41436.80     41342.00
      4194304           10     70550.10     70943.20     70746.65

#----------------------------------------------------------------
# Benchmarking Allgather 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.16        12.16        12.16
            1         1000        12.61        12.61        12.61
            2         1000        12.52        12.52        12.52
            4         1000        12.41        12.41        12.41
            8         1000        12.69        12.69        12.69
           16         1000        12.71        12.71        12.71
           32         1000        13.26        13.27        13.26
           64         1000        13.06        13.06        13.06
          128         1000        17.72        17.73        17.72
      2097152           20     23349.75     23350.30     23350.03
      4194304           10     38150.00     38151.60     38150.80

#----------------------------------------------------------------
# Benchmarking Allgatherv 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.48        12.48        12.48
            1         1000        12.41        12.42        12.42
            2         1000        12.16        12.17        12.16
            4         1000        12.21        12.21        12.21
            8         1000        12.52        12.52        12.52
           16         1000        12.35        12.35        12.35
           32         1000        13.09        13.09        13.09
           64         1000        12.80        12.80        12.80
          128         1000        17.19        17.19        17.19
      2097152           20     19057.95     19058.55     19058.25
      4194304           10     37964.00     37965.39     37964.70

#----------------------------------------------------------------
# Benchmarking Alltoall 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.44        12.44        12.44
            1         1000        12.72        12.72        12.72
            2         1000        12.50        12.50        12.50
            4         1000        12.56        12.56        12.56
            8         1000        12.56        12.56        12.56
           16         1000        12.75        12.75        12.75
           32         1000        12.86        12.86        12.86
           64         1000        13.73        13.73        13.73
          128         1000        17.87        17.87        17.87
      2097152           20     19927.10     19927.60     19927.35
      4194304           10     39608.10     39609.79     39608.94

#----------------------------------------------------------------
# Benchmarking Bcast 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.09         0.09         0.09
            1         1000         9.09         9.09         9.09
            2         1000         9.07         9.08         9.07
            4         1000         9.06         9.07         9.07
            8         1000         9.09         9.10         9.09
           16         1000         9.15         9.16         9.16
           32         1000         9.44         9.44         9.44
           64         1000         9.56         9.57         9.56
          128         1000        11.63        11.64        11.63
      2097152           20      8740.05      8740.20      8740.12
      4194304           10     17313.69     17313.90     17313.80

#---------------------------------------------------
# Benchmarking Barrier 
# ( #processes = 2 ) 
#---------------------------------------------------
 #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
         1000        12.45        12.45        12.45


/////////////////////////// 860 //////////////////////////////////


#---------------------------------------------------
# Benchmarking PingPong 
# ( #processes = 2 ) 
#---------------------------------------------------
       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000         8.93         0.00
            1         1000         9.14         0.10
            2         1000         9.17         0.21
            4         1000         9.14         0.42
            8         1000         9.41         0.81
           16         1000         9.54         1.60
           32         1000         9.85         3.10
           64         1000        10.06         6.06
          128         1000        12.77         9.56
      2097152           20     11924.45       167.72
      4194304           10     23752.65       168.40

#---------------------------------------------------
# Benchmarking PingPing 
# ( #processes = 2 ) 
#---------------------------------------------------
       #bytes #repetitions      t[usec]   Mbytes/sec
            0         1000        11.94         0.00
            1         1000        12.47         0.08
            2         1000        12.83         0.15
            4         1000        13.02         0.29
            8         1000        12.41         0.61
           16         1000        12.82         1.19
           32         1000        12.07         2.53
           64         1000        12.27         4.98
          128         1000        14.50         8.42
      2097152           20     21075.20        94.90
      4194304           10     42104.29        95.00

#-----------------------------------------------------------------------------
# Benchmarking Sendrecv 
# ( #processes = 2 ) 
#-----------------------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]   Mbytes/sec
            0         1000        12.20        12.21        12.20         0.00
            1         1000        12.83        12.84        12.83         0.15
            2         1000        12.85        12.85        12.85         0.30
            4         1000        12.61        12.62        12.62         0.60
            8         1000        12.63        12.63        12.63         1.21
           16         1000        12.55        12.55        12.55         2.43
           32         1000        12.40        12.40        12.40         4.92
           64         1000        12.70        12.70        12.70         9.61
          128         1000        14.54        14.55        14.55        16.78
      2097152           20     21075.50     21075.85     21075.67       189.79
      4194304           10     42110.90     42112.10     42111.50       189.97

#-----------------------------------------------------------------------------
# Benchmarking Exchange 
# ( #processes = 2 ) 
#-----------------------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]   Mbytes/sec
            0         1000        21.23        21.23        21.23         0.00
            1         1000        21.56        21.63        21.59         0.18
            2         1000        21.56        21.57        21.56         0.35
            4         1000        21.49        21.49        21.49         0.71
            8         1000        21.63        21.63        21.63         1.41
           16         1000        21.68        21.68        21.68         2.81
           32         1000        21.87        21.88        21.88         5.58
           64         1000        22.16        22.16        22.16        11.02
          128         1000        24.66        24.66        24.66        19.80
      2097152           20     42154.20     42155.05     42154.62       189.78
      4194304           10     84224.61     84225.20     84224.90       189.97

#----------------------------------------------------------------
# Benchmarking Allreduce 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.15         0.16         0.16
            4         1000        19.93        19.94        19.94
            8         1000        20.31        20.33        20.32
           16         1000        20.77        20.78        20.78
           32         1000        21.54        21.55        21.55
           64         1000        21.96        21.97        21.96
          128         1000        26.05        26.06        26.05
      2097152           20     36295.15     36300.15     36297.65
      4194304           10     72057.59     72060.60     72059.09

#----------------------------------------------------------------
# Benchmarking Reduce 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.09         0.09         0.09
            4         1000        10.48        10.49        10.49
            8         1000        10.92        10.93        10.92
           16         1000        11.03        11.04        11.03
           32         1000        11.40        11.41        11.40
           64         1000        11.65        11.66        11.65
          128         1000        13.85        13.86        13.86
      2097152           20     24145.65     24442.65     24294.15
      4194304           10     47357.39     48542.51     47949.95

#----------------------------------------------------------------
# Benchmarking Reduce_scatter 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.65         0.66         0.66
            4         1000        22.04        22.05        22.05
            8         1000        22.72        22.73        22.73
           16         1000        23.31        23.32        23.31
           32         1000        23.89        23.90        23.90
           64         1000        24.45        24.46        24.45
          128         1000        26.94        26.95        26.94
      2097152           20     33828.60     33844.10     33836.35
      4194304           10     67314.90     67377.90     67346.40

#----------------------------------------------------------------
# Benchmarking Allgather 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.16        12.16        12.16
            1         1000        12.32        12.32        12.32
            2         1000        12.33        12.33        12.33
            4         1000        12.36        12.36        12.36
            8         1000        12.61        12.61        12.61
           16         1000        12.71        12.71        12.71
           32         1000        13.04        13.04        13.04
           64         1000        13.86        13.86        13.86
          128         1000        17.59        17.59        17.59
      2097152           20     26607.50     26608.05     26607.78
      4194304           10     53338.10     53338.91     53338.50

#----------------------------------------------------------------
# Benchmarking Allgatherv 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.18        12.18        12.18
            1         1000        12.35        12.35        12.35
            2         1000        12.30        12.30        12.30
            4         1000        12.32        12.32        12.32
            8         1000        12.52        12.53        12.53
           16         1000        12.91        12.91        12.91
           32         1000        13.11        13.11        13.11
           64         1000        13.73        13.73        13.73
          128         1000        17.58        17.58        17.58
      2097152           20     26836.70     26838.00     26837.35
      4194304           10     53090.61     53091.80     53091.20

#----------------------------------------------------------------
# Benchmarking Alltoall 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000        12.86        12.86        12.86
            1         1000        12.85        12.85        12.85
            2         1000        13.19        13.19        13.19
            4         1000        13.01        13.01        13.01
            8         1000        13.23        13.23        13.23
           16         1000        13.43        13.44        13.44
           32         1000        13.78        13.78        13.78
           64         1000        14.41        14.41        14.41
          128         1000        18.18        18.18        18.18
      2097152           20     27169.85     27170.25     27170.05
      4194304           10     54303.90     54304.40     54304.15

#----------------------------------------------------------------
# Benchmarking Bcast 
# ( #processes = 2 ) 
#----------------------------------------------------------------
       #bytes #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
            0         1000         0.07         0.10         0.08
            1         1000         9.29         9.30         9.29
            2         1000         9.23         9.24         9.24
            4         1000         9.25         9.26         9.26
            8         1000         9.54         9.55         9.55
           16         1000         9.69         9.69         9.69
           32         1000         9.96         9.98         9.97
           64         1000        10.14        10.15        10.15
          128         1000        12.21        12.22        12.21
      2097152           20     11937.15     11937.50     11937.32
      4194304           10     23764.00     23764.71     23764.35

#---------------------------------------------------
# Benchmarking Barrier 
# ( #processes = 2 ) 
#---------------------------------------------------
 #repetitions  t_min[usec]  t_max[usec]  t_avg[usec]
         1000        12.90        12.90        12.90






More information about the Beowulf mailing list