[Beowulf] Re: newbie's dilemma / firewire? (Hahn)

[Mark Hahn]

> >> effective in some cases. However, the bulk file transfer rates for  
> >> very large files via FireWire 400 (1394a) is significantly faster  
> >> than 1000baseT and FireWire 800 (1394b) is more than 240% faster  
> >> that  GigaBit EtherNet.
> >
> > can you show some references for this?  it's hard to understand why  
> > a 400 Mbps connection would be "sigificantly faster" than a 1000 Mbps
> > connection.  or are you talking mainly about shortcomings in some  
> > platform's drivers/stack?
> 
> Easiest, simplest reference is this image: http://unibrain.com/ 
> products/assets/FireNET5.jpg

thanks.  to me, it looks like the device can manage at most ~24 MB/s,
and both GBE and FireNet can saturate it.  that's not too surprising,
since GBE's peak theoretical is 125 MB/s, and if you're getting less
than say 60, you're doing something wrong (85 is reasonable.)

>  ... or read the whole report at bottom  
> of: http://unibrain.com/products/driverapi/firenet.htm

certainly appears to be saturating because of the disk,
not the interconnect.  a P4/1.7 is circa -5 years, and in 
those days 23.4 MB/s wasn't atrocious.  now, ~50 is about
the least you can expect from a single disk, and that's 
not enough to saturate GBE.

> Note that for large files, the FireWire network bulk file transfer  
> rates can exceed GigaBit performance.

definitely not shown on those pages...

> The above reports (arguably  
> promotional) are for FireWire 1394a (400 Mbits / second). When  
> similar tests are run on FireWire 1394b (800 Mbits / second),  
> performance can exceed 240% of network bulk file transfer (write  
> times) of GigaBit EtherNet.

well, that's what you said before, and I'm still asking for some 
indication its true.

I'm mainly curious about the 240% part.  oh, wait, are you just 
dividing 24 MB/s by 10 MB/s in the table at the bottom?  that's 
silly.

> Why is it comparable to or faster from a processor running at around  
> 40% of GigaBit processor speeds? Processor efficiency FireWire has a  
> 32-bit "risc" type microprocessor, is peer to peer in hardware /  
> firmware and has other lean architecture features. (Small address  
> space, data frame large = more efficient data packet over double  
> duplex connections.)

that's nice marketing-speak, but content-free, I'm afraid.

> Future: there are prototypes of FireWire 1600 and 3200 operating over  
> fiber and using multiple "colors" ... using two fibers (duplex),  
> multiplexed by frequency discrimination (4 or 8 channels) using the  
> same FireWire 800 (Texas Instruments chips) ... comparable to ~~  
> 64000baseT ... and the involved engineers say they can stay ahead of  
> Moore's Law beyond several more years.

OK, I get the point.  though I have to say I really don't expect optical
WDM firewire to make any difference ;)

what's the latency of a typical 1394 switch?  are they all store/forward?
are they all full wirespeed (in internal bandwidth)?  what's the max fanout?

I'd personally love to see something better in every way than GBE/10GE,
but just don't see it happening.  no, please, don't suggest IB :)