[Beowulf] Intel buys QLogic InfiniBand business
diep at xs4all.nl
Fri Jan 27 13:42:24 PST 2012
On Jan 27, 2012, at 9:19 PM, Joe Landman wrote:
> On 01/27/2012 03:06 PM, Vincent Diepeveen wrote:
>> On Jan 27, 2012, at 8:29 PM, Håkon Bugge wrote:
>>> On 23. jan. 2012, at 20.55, Greg Lindahl wrote:
>>>> On Mon, Jan 23, 2012 at 11:28:26AM -0800, Greg Lindahl wrote:
>>>> I figured out the main why:
>>>>> Server-class 10Gb Ethernet Adapter and LOM revenues have recently
>>>>> surpassed $100 million per quarter, and are on track for about
>>>>> percent annual growth, according to Crehan Research.
>>>> That's the whole market, and QLogic says they are #1 in the FCoE
>>>> adapter segment of this market, and #2 in the overall 10 gig
>>>> market (see
> I found that statement interesting. I've actually not known anything
> about their 10GbE products. My bad.
>>> That can explain why QLogic is selling, but not why Intel is buying.
>>> 10 years ago, Intel went _out_ of the Infiniband marked, see http://
>>> So has the IB business evolved so incredible well compared to what
>>> Intel expected back in 2002? Do not think so.
>>> I would guess that we will see message passing/RDMA over
>>> Thunderbolt or similar.
> Intel buying makes quite a bit of sense IMO. They are in 10GbE
> and NICs, and being in IB silicon and HCAs gives them not only a hedge
> (10GbE while growing rapidly, is not the only high performance network
> market, and Intel is very good at getting economies of scale going
> its silicon ... well ... most of its silicon ... ignoring Itanium here
> ...). Its quite likely that Intel would need IB for its PetaScale
Why buy previous generation IB in such case?
It's about the ethernet of course...
They produce tens of millions of cpu's each quarter and also
announced a SoC (socket on chip).
From SoC's actually the market produces billions a year. So it's
alucrative market, yet highly competative.
Having 10 gigabit ethernet on such SoC and the total at a low price
would give intel a huge lead there
worth dozens of billions a year.
It's not clear to me where all their SoC plans go, but i bet right
now they are open to any market needing SoC's.
Note that many SoC's are dirt cheap. Even in very low volume we speak
about some tens of dollars, cpu included
and other connectivity included.
Price is everything there, yet i guess intel will be offering the
'top' SoC's there with faster cpu's and 10 GigE.
Then they produce a bunch of mainboards.
Think also of upcoming generation of consoles, ipad 3's and similar
products etc - it's not clear
yet which company gets the contracts for upcoming consoles, it's all
wide open for now.
Yet they might sell also a 100+ million of those.
Intel is an attractive company to do business with for console
IBM's cell kind of lost momentum there and has nothing new to offer
that really outperforms as it seems.
Also power usage of cell was kind of disappointing.
Initial version PS3 was 220 watts on average and 100% usage it could
go up to 380+ watt.
Try to put that on your couch.
Don't confuse this with the later crunching CELL version, a much
improved chip, used for some supercomputers.
Yet if i remember well, some reports, was it Aad v/d Steen (?)
already predicted it would be not interesting for upcoming
as it is some kind of hybrid chip - which has no long term future.
He was right.
> plans. Someone here postualted putting the silicon on the CPU. Not
> sure if this would happen, but I could see it on an IOH, easily. That
> would make sense (at least in terms of the Westmere designs ... for
> Romley et al. I am not sure where it would make most sense).
> But Intel sees the HPC market growth, and I think they realize that
> there are interesting opportunities for them there with tighter high
> performance networking interconnects (Thunderbolt, USB3, IB, 10GbE
> native on all these systems).
Undoubtfully they'll try something in the HPC market.
If you already have put lots of cash in development of a product it's
better to put it
on the market.
Based upon their name they'll sell some.
And some years from now they should have something bigtime improved.
Yet realize how complicated it is to tape out a GPU at a new process
if you aren't sure you gonna sell a 100+ million of them.
Such massive projects have to pay back for factories. A product
that's having a potential of not even selling for over a few dozens
of billions of dollars is not even interesting to develop.
Just startup costs for a GPU at a new proces technology is some
dozens of millions for each run and the more complex it is and the
newer the proces technology the more expensive it is.
Realize IBM produces its power7 and bluegene/q upcoming cpu at 45 nm
GPU's release now in 28 nm. That's giving theoretically an advantage
of a tad less of (45 / 28) ^ 2 = 2.58
So a gpu of intel needs to be factor 2.58 better in the same proces
technology than todays gpu's of
AMD (already released 28 nm) and Nvidia (coming soon 28 nm i'd expect).
This where with cpu's, intels big advantage is always that they are
better in getting newer proces technologies to work sooner than the
Ivy Bridge will be 22 nm so i heard rumours.
>> Qlogic offers that QDR.
>> Mellanox is a generation newer there with FDR.
>> Both in latency as well as in bandwidth a huge difference.
> Haven't looked much at FDR or EDR latency. Was it a huge delta (more
> than 30%) better than QDR? I've been hearing numbers like 0.8-0.9 us
> for a while, and switches are still ~150-300ns port to port. At some
Posting here some months ago from Gilad Shainer was it's 0.85 us RDMA
for FDR versus 1.3 us or so for the other;
more importantly for clusters is the bandwidth.
I guess that pci-e 3.0 allows simply much higher speeds whereas the
QDR is PCI-E 2.0 stuff.
Isn't pci-e 3.0 about 2x higher bandwidth than 2 pci-e 2.0?
Now i might be happy with that last, but i guess that for big FFT's
or be it matrice,
you still need massive bandwidth.
Even if n is big in O ( k * n log n )
Where k in case of matrice is a tad bigger than n and in case of
Number Theory is usually around the number of bits,
so 3.32 times n or so, that means you still need k steps of n log n.
That's massive bandwidth.
> point I think you start hitting a latency floor, bounded in part by
> but also by an optimal technology path length that you can't shorten
> without significant investment and new technology. Not sure how close
> we are to that point (maybe someone from Qlogic/Mellanox could comment
> on the headroom we have).
There is a lot of headroom for better latencies from software viewpoint,
as cpu's keep getting faster yet latency of years ago networks was
worse than what's there now.
In case of hardware i really am no expert there.
> Bandwidth wise, you need E5 with PCIe 3 to really take advantage of
> So again, its a natural fit, especially if its LOM ....
All the socket2011 boards that are in the shops now are PCI-e 3.0 and
a wave of
mainboards with 2 sockets will release a few days before or at the
same day that
intel finally releases the Xeon version of Sandy Bridge.
Seems it didn't release yet as it's not too high clocked, if i look
at this sample cpu :)
It's 2Ghz to be precise (8 cores Xeon).
> Curiously, I think this suggests that ScaleMP could be in play on the
> software side ... imagine stringing together bunches of the LOM FDR/
> motherboards with E5's and lots of ram into huge vSMPs (another
> Shai may tell me I'm full of it (hope he doesn't), but I think
> this is
> a real possibility. The Qlogic purchase likely makes this even more
> interesting for Intel (or Cisco, others as a defensive acq).
A technology that just sold to 300 machines, this is not interesting
market for intel.
They have very expensive factories that each cost many billions of
These need to produce nonstop and sell products, to pay back for the
factories and to make a profit.
Intel used to be worth over a 100 billion dollar at NASDAQ.
Wasting your most clever engineers, from which each company always
has too few, to products that can't keep busy your
factories, is a total waste of time. So your huge base of B-class
engineers, let me not quote some mailing list names,
that's the ones you move to Qlogic then for the HPC.
That's enough to keep it afloat for a while in combination with
Intels profit is too huge to be busy toying with tiny markets with a
handful of customers,
from which majority forgot to take their medicine when you propose
rewriting the software to some new hardware platform
you are gonna unroll. A habit intel is not exactly excited about of
course, as they like to sell each time new technology.
Also each larrabee intel would sell means they sell a bunch of xeons
less of course.
> We sure do live in interesting times!
Not for everyone i guess - many lost their job and as i predicted
some years ago a guy with a
nobel prize might be carpet bombing a huge nation this summer.
Intel has 3 huge factories in Israel last time i checked.
It sure can give unpredicted results for future.
> Joseph Landman, Ph.D
> Founder and CEO
> Scalable Informatics Inc.
> email: landman at scalableinformatics.com
> web : http://scalableinformatics.com
> phone: +1 734 786 8423 x121
> fax : +1 866 888 3112
> cell : +1 734 612 4615
> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
> To change your subscription (digest mode or unsubscribe) visit
More information about the Beowulf