[Beowulf] massive parallel processing application required

[Jim Lux]

At 07:40 AM 2/1/2007, Mark Hahn wrote:
>>Not true.  Distributed computing is more and more mainstream.  I think too
>
>oh, one other snide comment about grid: I suspect the grid-fad could 
>not have happened without the fraud perpetrated by worldcom and others during
>the internet bubble.  in those days, it was popular to claim that the network
>was becoming truely ubiquitous and incomprehensibly fast.  for instance:
>
>http://www-128.ibm.com/developerworks/grid/library/gr-heritage/#N100A6



In the long run, ubiquitous and fast IS going to be true (however, 
latency is something you can't get around... speed of light and all 
that).  As long ago as 1993, I was at a conference where a speaker 
from AT&T commented that historical telecom pricing methods (longer 
distances cost more) were obsolete, since the dominant cost was in 
the termination, with, even then, a gross oversupply of fiber across 
the Atlantic.  Hence the availability of cheap flat rate long 
distance (5c a minute anywhere, anytime).. the bulk of the system is 
no longer capacity limited.


>I don't know about you, but in the 6 years since then, my home net 
>connection has stayed the same speed, possibly a bit more expensive.

Interestingly, they've just rolled out FiOS (fiber to the home) in my 
area, which is a HUGE jump in potential bandwidth from the existing 
DSL or Cable Modem delivery methods.  And, moderately competitive in 
price (5 Mbps is $40/month, including the bundled ISP kinds of 
features).  What's fascinating is the faster tiers.. you can get 15 
Mbps down/2 up for $50/mo and 30 M down/5 up for $180

Granted, these are consumer offerings and have all the usual network 
congestion caveats, but hey, at least they are offering 30 Mbps for 
the last mile, which is quite impressive.


>desktop/LANs are still mostly at 100bT, with 1000bT in limited use.

But that's more driven by replacement cycles and the lack of real 
demand for faster speeds to the desktop.  If your facility has a 1.5 
Mbps pipe to the internet, giving users a 1 Gb/s won't change their 
performance much compared to 100 Mb/s.  There's also a wiring 
infrastructure issue.  While desktops are typically replaced on a 3 
year cycle, the wiring infrastructure cycles through a bit slower, 
especially in smaller businesses and residential (that is, I'm not 
likely to start ripping out the drywall to replace the Cat 5 wiring I 
put in back in 1998)... and frankly, since right now, I have maybe 
700 kbps at home to the internet (one way), and then a wireless 
connection from laptop to home network, there's not much to be gained 
by improving the home wiring infrastructure.  (If I go with the FiOS 
offering though, that may prompt some re-evaluation)

Likewise, a small business with half a dozen or a dozen desktops and 
a couple servers isn't going to see a huge benefit from faster 
networking, because they're throttled by the server's disk speed, 
more than anything else. (assuming they're not hosting a big website, etc.)

So, you're looking at GigE making a difference in two 
areas:  replacing cable TV (all those 20 Mbps HDTV streams) and in 
big companies.  But even in big companies, GigE to the desktop 
doesn't necessarily buy you much, if you're all competing for the 
same server resources.

>I do notice that grabbing large files off the net (ftp, RPMs, etc)
>often runs at O(MBps) which is about a 10x improvement over the past 
>10-15 years.  so the doubling time turns out to be more like 3 years
>rather than 9 months.

Which is probably consistent with equipment refurbishment cycles.



>   in-cluster networking has improved somewhat faster, but not 
> dramatically so.
>_______________________________________________
>Beowulf mailing list, Beowulf at beowulf.org
>To change your subscription (digest mode or unsubscribe) visit 
>http://www.beowulf.org/mailman/listinfo/beowulf

James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
tel: (818)354-2075
fax: (818)393-6875