[Beowulf] massive parallel processing application required
James.P.Lux at jpl.nasa.gov
Thu Feb 1 16:56:12 PST 2007
At 03:25 PM 2/1/2007, Mark Hahn wrote:
>>>the internet bubble. in those days, it was popular to claim that
>>>was becoming truely ubiquitous and incomprehensibly fast. for instance:
>>In the long run, ubiquitous and fast IS going to be true (however, latency is
>in the long run, everything is true ;)
>>gross oversupply of fiber across the Atlantic. Hence the
>>availability of cheap flat rate long distance (5c a minute
>>anywhere, anytime).. the bulk of the system is no longer capacity limited.
>interesting - I assumed that long-distance became cheap not due to
>oversupply of fiber and bandwidth, but rather transition away from
>old-fashioned circuit switching (ie, towards digital compressed voice
Not much compression going on for voice traffic. It's carried as 64
kbps data at 8 ksamples/second, pretty much. There is some
statistical multiplexing possible (TASI) because people don't talk at
100% duty cycle, but not a huge amount.
>I know that buying fiber/lambdas/bandwidth is still very much not
>what I'd call cheap, though I have no doubt it's much better/cheaper
>than in the past.
In 1993, the capital cost of "one voice channel" worth (64 kbps) of
capacity across the atlantic was less than $10, as I recall.
Compare that to leased line T-1 rates back then of many dollars per
eighth of a mile per month, and that's before you bought the CSU/DSU
to connect to the copper.
Mind you, the ATT guy thought that it would be 155 Mbps ATM to the
desktop, and we see where that went.
>>>I don't know about you, but in the 6 years since then, my home net
>>>connection has stayed the same speed, possibly a bit more expensive.
>>Interestingly, they've just rolled out FiOS (fiber to the home) in
>>my area, which is a HUGE jump in potential bandwidth from the
>>existing DSL or Cable Modem delivery methods. And, moderately
>>competitive in price (5 Mbps is $40/month, including the bundled
>>ISP kinds of features). What's fascinating is the faster tiers..
>>you can get 15 Mbps down/2 up for $50/mo and 30 M down/5 up for $180
>seems strange to me - what kind of residential customer would pay
>for that kind of thing (and remain free of the RIAA/MPAA)?
An interesting question.. I think the upper tiers are there to
complement similar offerings in the commercial/business market. Or,
there IS a burgeoning market for live video feeds from adult
entertainment providers. Without them, the VCR market would never
have taken off. Contemplate Youtube type applications, but in HD..
20 Mbps is the basic rate for HD. I might be interested, for
instance, in seeing the Mentos and Pepsi artists in HD, rather than
lowfi 15 fps QCIF.
But, also, consider something like streaming audio at CD quality (not
MP3 compressed).. A stereo 44.1ksps 16 bit stream is about 1.5 Mbps,
and say I, my wife, and my daughters all want to listen to different
programs at the same time.
There will also be video that is not afflicted by MPAA. NasaTV is
free to all and streamed over the network as well as being shoved out
over C-band transponders.
I can see using 15M+ sorts of rates in bursts for myself (downloading
the aforementioned climate databases, for instance...)
>some smart form of wireless seems like an obvious good solution for
>residential last-mile. maybe that's a disruptive innovation that will
>finally put the telco/cableco's out of their misery.
Nobody has come up with a *good* wireless solution that is as cheap
and reliable as pulling a physical media. There's a raft of spectrum
occupancy issues, etc.
Let's assume you've got a neighborhood with 400 houses in it at a
density of, say, 500 square meters/house (roughly 8
houses/acre). It's perhaps 10-20 meters between houses on
average. Say each house needs 50 Mbps of bandwidth (e.g. two cable
channels worth). If you use a short range wireless scheme (notional
range of 50 m) a given transmitter is going to cover half a dozen
houses, so each transmitter would need a bandwidth of about 300 Mbps
(which is fairly hefty, but not out of the question). AND there
would need to be some smart switching in the system that feeds that
transmitter the correct subset of the Terabits/second
available... And, some way to cleverly do spectrum reuse (so that if
you have houses A, B,C, D, and E lined up, A can use channel 1, B can
use channel 6, C can use channel 11, and by house D, the signal for
Channel 1 going to house A has faded enough that we can reuse it for
D, 6 for E, etc.) This is highly nontrivial, and nobody has come up
with a automagic way to do it that is efficient and self organizing.
Right now, though, the Cable TV folks feed 1 GHz of bandwidth to you
and YOU do the channel selection, which reduces their physical plant
cost... all they need is power distribution with no intelligence,
just management of SNR. (this breaks down in the upstream case,
which is a fundamental problem with Cable Modems)
>>Likewise, a small business with half a dozen or a dozen desktops
>>and a couple servers isn't going to see a huge benefit from faster
>>networking, because they're throttled by the server's disk speed,
>>more than anything else.
>if their servers disks are only 100bT speed, they're broken. it may well
>be that most SMB servers are that crappy, in spite of the fact that
>a recycled linux box and one disk will deliver 40 MB/s...
Not so much a limitation as that, as the 10 desktops aren't all going
to be hitting the server at exactly the same time, most of the
time. Relatively few business desktops are doing things like
streaming video. They're just moving documents to and from the
server, and that's a sort of bursty traffic, so its not a big
deal. And 40 MB/s implies 13 Megatransfers/second across a 32 bit
bus, with a 33 MHz bus, a transfer from the disk and a transfer to
the NIC doesn't leave a whole lot of time for fetching instructions
from RAM, etc.
now, if your office is comprised of diskless clients....that's another story.
>>So, you're looking at GigE making a difference in two
>>areas: replacing cable TV (all those 20 Mbps HDTV streams)
>how many 20Mb streams does a typical endpoint need? either residential
I can see at least 3 streams for residential. 1 for live viewing, 1
for recording on the TIVO, 1 for the second TV.
>>and in big companies. But even in big companies, GigE to the
>>desktop doesn't necessarily buy you much, if you're all competing
>>for the same server resources.
>wow, dim view of the competence of server admins, but you may be right...
No.. it's that network traffic from desktop to server just isn't all
that high in most environments. For instance, I consume almost NO
network bandwidth most of the time at work, because most of what I
work with is on the local machine.
Even in a high transaction rate call center, there's just not that
many bytes flying back and forth. "yes, Mr. Lux, and your account
number is? ...." blurp there's 100 bytes to the server in a SQL
query, and maybe a kilobyte coming back.
10 seconds pass,
"And you'd like the bullion delivered where?"
blurp.. after 30 seconds, the operator sends the delivery address
with a few hundred bytes to the transaction processor.
blurp...100 bytes come back
"your confirmation number is 2.71828, Thank you for calling"
Then, that triggers a few more kbytes of traffic to the vault and the
delivery truck company, etc.
But overall, that's what, an average of 1 kb per second, at most? So
the call center has 1000 people.. we're up to only 1 Megabit/second.
Even if they do complete screen paints at every step over the
network, it's still not that much traffic.
Some sort of call center where they look at scanned images might be
an example of a bigger volume user. "Yes.. I'm looking at your
bearer bonds now, and we'll be able to execute that sell order for
100,000 shares MSFT." or, more realistically, "I'm looking through
your loan application now and on page 32, there's a problem with the
property description you submitted three years ago." or "Yes, Mr.
Lux, that IS a big dent that we need to fix in your bumper" But even
then, a full screen image is only a few megabytes, at most, unless
you're totally profligate with uncompressed 24 bit TIFF images.
The big advantage of GigE to the desktop is that when you do send big
files (say a full screen image), it takes less time. But the average
rate is still low.
>regards, mark hahn.
James Lux, P.E.
Spacecraft Radio Frequency Subsystems Group
Flight Communications Systems Section
Jet Propulsion Laboratory, Mail Stop 161-213
4800 Oak Grove Drive
Pasadena CA 91109
More information about the Beowulf