Tulip TCP Performance Problems on Compaq XP1000

Jie Chen chen@jlab.org
Mon Jun 7 16:17:24 1999


Hi,

    We recently purchased some of XP1000 machines from Compaq. These
machines have Digital Tulip Fast Ethernet
    card. We have installed Redhat 5.2 Linux with custom built kernel
2.2.5 (egcs 1.1.2).  Today I tried to use netpipe
    benchmark to find out how well TCP/IP performs under alpha/linux.
The following are my findings:

    netpipe sends data of multiple sizes from one node to another. Here
are the results:
    Latency: 0.000060
    Now starting main loop
    .........
    50:    2048 bytes    543 times --->    68.23 Mbps in 0.000229sec
    51:     2051bytes    546 times --->    67.83 Mbps in 0.000231sec
    52:    3069bytes    543 times  --->    70.63 Mbps in 0.000332sec
    53:    3072bytes    502 times --->      70.74Mbps in 0.000332sec
    54:    3075bytes    502 times --->    70.58Mbps in 0.000332sec
    55:    4093bytes    251 times --->    0.31 Mbps in 0.099608sec
    56:    4096bytes    7 times      --->    0.31 Mbps in 0.099602sec
     .......

    Clearly the TCP performence drops dramatically at transfering data
size around 4096 bytes. I have tried using
    different kernel (2.2.5 and 2.2.8) , diffrent version of tulip.c
(0.89h to 0.91e) and with or without fast ethernet switch.
    Could you give me some hint on this?  To help you to find problems,
I include some kernel messages and outputs
    from tulip-diag program.

    The following are the kernel messages when I did insmod tulip.o:
Jun  7 15:40:32 hpcdev01 kernel: tulip.c:v0.91e 5/27/99
becker@cesdis.gsfc.nasa.gov
Jun  7 15:40:32 hpcdev01 kernel: eth0: Digital DS21143 Tulip rev 65 at
0x100008000, 00:00:F8:71:5E:E9, IRQ 45.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  EEPROM default media type
Autosense.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #0 - Media 10baseT (#0)
described by a 21142 Serial PHY (2) block.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #1 - Media 10base2 (#1)
described by a 21142 Serial PHY (2) block.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #2 - Media AUI (#2)
described by a 21142 Serial PHY (2) block.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #3 - Media 10baseT-FD (#4)
described by a 21142 Serial PHY (2) block.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #4 - Media 100baseTx (#3)
described by a 21143 SYM PHY (4) block.
Jun  7 15:40:32 hpcdev01 kernel: eth0:  Index #5 - Media 100baseTx-FD
(#5) described by a 21143 SYM PHY (4) block.

    In addition, the output of tulip-diag are:

tulip-diag.c:v1.10 4/12/99 Donald Becker (becker@cesdis.gsfc.nasa.gov)
Index #1: Found a Digital DS21143 Tulip adapter at 0x8000.
 Port selection is 100mbps-SYM/PCS 100baseTx scrambler, full-duplex.
 Transmit started, Receive started, full-duplex.
  The Rx process state is 'Transferring Rx frame into memory'.
  The Tx process state is 'Closing Tx descriptor'.
  PCI bus error!: Unknown 7.  <------ Is this fatal ?
  The transmit unit is set to store-and-forward.
 Interrupt sources are pending!  CSR5 is ffffffff.
   Tx done indication.
   Tx complete indication.
   Tx out of buffers indication.
   Transmit Jabber indication.
   Link passed indication.
   Tx FIFO Underflow indication.
   Rx Done indication.
   Receiver out of buffers indication.
   Receiver stopped indication.
   Receiver jabber indication.
   Link changed indication.
   Timer expired indication.
   Link failed indication.
   PCI bus error indication.
   Early Rx indication.
 Use '-a' to show device registers,
     '-e' to show EEPROM contents,
  or '-m' to show MII management registers.


    The following are the output when I ran tulip-diag -a -f


tulip-diag.c:v1.10 4/12/99 Donald Becker (becker@cesdis.gsfc.nasa.gov)
Index #1: Found a Digital DS21143 Tulip adapter at 0x8000.
Digital DS21143 Tulip chip registers at 0x8000:
  ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
  ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
 Port selection is 100mbps-SYM/PCS 100baseTx scrambler, full-duplex.
 Transmit started, Receive started, full-duplex.
  The Rx process state is 'Transferring Rx frame into memory'.
  The Tx process state is 'Closing Tx descriptor'.
  PCI bus error!: Unknown 7.
  The transmit unit is set to store-and-forward.
 Interrupt sources are pending!  CSR5 is ffffffff.
   Tx done indication.
   Tx complete indication.
   Tx out of buffers indication.
   Transmit Jabber indication.
   Link passed indication.
   Tx FIFO Underflow indication.
   Rx Done indication.
   Receiver out of buffers indication.
   Receiver stopped indication.
   Receiver jabber indication.
   Link changed indication.
   Timer expired indication.
   Link failed indication.
   PCI bus error indication.
   Early Rx indication.

    The following are the outputs when I ran tulip-dia -e

tulip-diag.c:v1.10 4/12/99 Donald Becker (becker@cesdis.gsfc.nasa.gov)
Index #1: Found a Digital DS21143 Tulip adapter at 0x8000.
 Port selection is 100mbps-SYM/PCS 100baseTx scrambler, full-duplex.
 Transmit started, Receive started, full-duplex.
  The Rx process state is 'Transferring Rx frame into memory'.
  The Tx process state is 'Closing Tx descriptor'.
  PCI bus error!: Unknown 7.
  The transmit unit is set to store-and-forward.
 Interrupt sources are pending!  CSR5 is ffffffff.
   Tx done indication.
   Tx complete indication.
   Tx out of buffers indication.
   Transmit Jabber indication.
   Link passed indication.
   Tx FIFO Underflow indication.
   Rx Done indication.
   Receiver out of buffers indication.
   Receiver stopped indication.
   Receiver jabber indication.
   Link changed indication.
   Timer expired indication.
   Link failed indication.
   PCI bus error indication.
   Early Rx indication.
 This interface is missing the EEPROM.
  This is likely the non-primary interface on a multiport board.

    Moreover the following are the outputs for tulip-diag -m

tulip-diag.c:v1.10 4/12/99 Donald Becker (becker@cesdis.gsfc.nasa.gov)
Index #1: Found a Digital DS21143 Tulip adapter at 0x8000.
 Port selection is 100mbps-SYM/PCS 100baseTx scrambler, full-duplex.
 Transmit started, Receive started, full-duplex.
  The Rx process state is 'Transferring Rx frame into memory'.
  The Tx process state is 'Closing Tx descriptor'.
  PCI bus error!: Unknown 7.
  The transmit unit is set to store-and-forward.
 Interrupt sources are pending!  CSR5 is ffffffff.
   Tx done indication.
   Tx complete indication.
   Tx out of buffers indication.
   Transmit Jabber indication.
   Link passed indication.
   Tx FIFO Underflow indication.
   Rx Done indication.
   Receiver out of buffers indication.
   Receiver stopped indication.
   Receiver jabber indication.
   Link changed indication.
   Timer expired indication.
   Link failed indication.
   PCI bus error indication.
   Early Rx indication.
   No MII transceivers found!
  Internal autonegotiation state is 'Invalid state'.

    Thank you very much.

--
#####################################################
Jie Chen
Thomas Jefferson National Accelerator Facility
12000, Jefferson Ave.
Newport News, VA 23606

(757)269-5046 (office) (757)269-6248 (fax)
chen@jlab.org
#####################################################