[Beowulf] HFT - was Intels Luxurious Oil Bath

Wed Sep 12 07:27:23 PDT 2012

You still haven't got a clue is it?

Oh as a reminder, would you be prepared to do next exercise?

Please calculate for me for the past few years, including of course  
the crash of 2008
and the flashcrash, how many datapoints each machine needs to proces  
to simulate,
and also how much storage you need for that, and also whether you can  
do that with simple gigabit
ethernet or whether you'd prefer infiniband.

After doing that calculation on the past, which will give you the  
answer that you need
a couple of thousands of nodes with a big bandwidth network, then  
that is the past.

Present and future is that you also take into account that the new  
datacenters will be using
infiniband and that processing times from the exchange is  in the  
microseconds now rather than
milliseconds.

So that means a factor 100+ increase in datapoints you have to  
simulate - whereas everyone
already needed a couple of thousands of PC's (when using cheapskate  
pc's and not something else).

Do you agree that a trader active in all markets, it doesn't matter  
whether they're having 100 persons personnel
or 275000, in both cases they need the same infrastructure as they  
have the same amount of datapoints?

Thanks,
Vincent

On Sep 12, 2012, at 4:16 PM, holway at th.physik.uni-frankfurt.de wrote:

> I was just making the point that a finance houses modeling/simulation
> computers are not dependent on low latency connections to the  
> exchange.
> Only their trading systems which will not be more than a few machines.
>
> A UK bank/trading house could locate their simulation systems in  
> Iceland
> and just put their HFT systems in london.
>
> Bank HPC (Modeling / Simulation) is different from HFT.
>
> HFT has essentially nothing to do with HPC although the output of  
> HPC will
> help the HFT boys set the triggers and thresholds on their HFT  
> systems.
>
> Bank HPC along with traditional HPC is a tiny part of all the machines
> sold in the EU. Out of 2 million servers sold in 2011 how many are  
> being
> used for this purpose?
>
> Thanks,
>
> Andrew
>
>> It seems that you first wrote:
>>
>>> High Frequency Trading is actually a tiny amount of the world wide
>>> computer power. Far far less than HPC. This kit tends to be located
>>> very close to the exchange anyway. and is generally uninteresting in
>> t>erms of green IT.
>>
>>> I recently met with one of the technical directors of Deutsch Börse
>>> which is the major central european stock exchange.. They are just
>>> moving their central trading platform from some VMS clustery thing
>>> (before my time) to a Intel / Infiniband / RedHat cluster of no more
>>> than 200 nodes of which about half were gateway nodes (with  
>>> interface
>>> cards for external connections to the finance houses). Its all small
>>> time stuff.
>>
>>> So please cease debasing these discussions with 'Iceland will never
>>> have datacenters because you cant use it for HFT". Thou speaks from
>>> thyne arse :) HFT is far smaller than HPC which is tiny anyway."
>>
>> And only AFTER i posted that behind each trading machine in the
>> datacenter of exchanges and platforms
>> there is an entire analysis department with huge HPC clusters.
>>
>> Then 30 minutes later you post Royal Bank of Scotland, which was kind
>> of bankrupt
>> because they did do so bad, that they have got 10k servers doing the
>> analysis.
>>
>> Had you known this *before* doing the silly post which breathes that
>> you see HFT as a business that has nearly
>> nothing to do with HPC, you would've realized that
>> not seldom most nations in Europe have just 1 HPC center that
>> government operates,
>> and hundreds of HPC centers operated by traders.
>>
>> They are an ever growing force in the HPC.
>>
>> The larger HFT in terms of infrastructure has got  a larger HPC
>> infrastructure than goverment has in terms of crunching power.
>> Each single one of them.
>>
>> That makes them a rather large market for HPC to take a look at and
>> take them serious. With each exchange slowly moving
>> from slowish gigabit ethernet with round trip processing time in the
>> many milliseconds, the amount of datapoints a second will
>> dramatically increase during surges. That will have a dramatic impact
>> on the crunching power they need and with those
>> bandwidths it's 100% HPC what they do back in their analysis  
>> department.
>>
>> Now how they do their calculations, there is a as many solutions to
>> that as there are traders, so there is not a sensible
>> word to say about it, except that they use lots of hardware and we
>> definitely can see this as HPC type calculations,
>> whether you like it or not.
>>
>> On Sep 10, 2012, at 2:13 PM, Andrew Holway wrote:
>>
>>>> That's very interesting! Where do you find out information on the
>>>> banks' setups?
>>>> The few times I have interviewed in the City they wouldn't let me
>>>> see into the server rooms.
>>>
>>> I just know a bit about RBS setup as I was interested in working for
>>> them a little while back. I've recently learnt that commerzbank has
>>> some kind of similar setup in Frankfurt.
>>
>> Why do i keep getting the impression you don't know the difference
>> between a trader and a bank?
>>
>>>
>>>
>>> I dont know about the others however. Although they are really not
>>> *that* secretive about it.
>>
>> Can you find for me of one of the larger High Frequency trading
>> hedgefund in the USA, which is called Renaissance Technologies,
>> their OFFICIAL homepage and then explain to me what infrastructure
>> they got?
>>
>> If they're totally open i bet they like to tell on their homepage
>> what hardware they got?
>>
>> That's just 1 out of the thousands.
>>
>> Actually nowadays they have a homepage. www.rentec.com :)
>>
>> Thanks,
>> Vincent
>>
>>>
>>>> I live next to Canary Wharf. There are some pretty tasty air
>>>> conditioning setups there,
>>>> if you look up on the roofs, and I should imagine some pretty big
>>>> server rooms.
>>>
>>> Who knows what they are cooling under there.
>>>
>>>>
>>>>
>>>>
>>>> The contents of this email are confidential and for the exclusive
>>>> use of the intended recipient.  If you receive this email in error
>>>> you should not copy it, retransmit it, use it or disclose its
>>>> contents but should return it to the sender immediately and delete
>>>> your copy.
>>>> _______________________________________________
>>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>>> Computing
>>>> To change your subscription (digest mode or unsubscribe) visit
>>>> http://www.beowulf.org/mailman/listinfo/beowulf
>>> _______________________________________________
>>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin
>>> Computing
>>> To change your subscription (digest mode or unsubscribe) visit
>>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>> _______________________________________________
>> Beowulf mailing list, Beowulf at beowulf.org sponsored by Penguin  
>> Computing
>> To change your subscription (digest mode or unsubscribe) visit
>> http://www.beowulf.org/mailman/listinfo/beowulf
>>
>
>