High Frequency Trading Conferences

I've been kindly asked to speak at two conferences this year: The first is HIFREQ 2011 on February the 24th. I'm on the panel with Prof Dave Cliff talking about next generation tech for HFT.

I'm also speaking at The High Frequency Trading World Conference in Amsterdam on the 7-9th June 2011.

Things I'll be talking about:

• The latest technology for market data acquisition and parsing, both in hardware and techniques using parallelised software.
• The role of FPGA, DSP and Memristors
• The future of XML
• Shared and reflective memory for market fusion
  
• CPU instructions for vectorisation and IO parallelism
• CPU analysis tools
• Kernel techniques for high performance
• Global layer 2 networks and VPLS
• Multicast routing in software
• Differential trading
• High precision global time
• The importance of platform analytics

Very grateful for both opportunities as I don't get out much :-)

Google's Strategy

I listened to a series of predictions by Mark Anderson of Strategy News Service several nights ago on Global Business during a bout of insomnia and was intrigued by his prediction that "Google has lost its way", "doesn't know what business they are in" and were "a rudderless organisation".

With regard to what business Google are in, my analysis is that they are in the data business: collection, analysis, enrichment and dissemination. The fact that they make money out of ads is coincidental and is indeed predicated on this, without which, targeting of ads would be impossible. The fact that they do it so successfully gives this strategy provenance.

With regards strategic direction, I can see where Mark Anderson is coming from, having done and MBA at the OU and the Certificate in Company Direction at the IOD. He's talking about the classic model of top-down corporate strategy development (which is rudimentary and heuristic.)

Strategic direction is usually driven by external marketing campaigns and salesmanship rather than the day-to-day expertise of the workforce as it's much easier dealing with external consultants, vendors and pundits. You do pay for their opinion after all and you're not accountable to them but, most of all, they'll tell you what you want to hear or sell you what their paymasters tell them to and buy you a nice lunch too.

The fact that, in general, the workforce is ignored and their tacit knowledge is never made explicit, analysed, persisted and incorporated into strategic direction is the sad state of knowledge management. It is mostly due to fear of the workforce and their expertise together with a fear of failure.

Perhaps Google is different (I don't know as I've neither worked there nor talked to any of their employees) in that they are driven by the expertise of their developers and develop products which best fit the data they have. Strategy may be bottom up and may be derived from the expertise of the many clever people who work there. Does this make them rudderless? Perhaps but the "rudder" developed by most organisations is often generic at best and irrelevant in practise. I think I'd rather go with my perception of how Google develops strategy

Visualisation Tools

Elie has sent me links to two tools - Tulip and Circos. There's a video of Tulip in action here.



This is a useful tool for visualisation of trades and market data and has proved invaluable.

A variant of the Circos approach was used quite effectively to replay a time series of equities trades by sector to end users showing their P and L and risk. Today it's used for multi-venue market visualisation.

Layers are for Cakes - Not Software

This erudite quote comes from Performance Anti-patterns by Bart Smaalders:

SOFTWARE LAYERING

Many software developers become fond of using layering to provide various levels of abstraction in their software. While layering is useful to some extent, its incautious use significantly increases the stack data cache footprint, TLB (translation look-aside buffer) misses, and function call overhead. Furthermore, the data hiding often forces
either the addition of too many arguments to function calls or the creation of new structures to hold sets of arguments. Once there are multiple users of a particular layer, modifications become more difficult and the performance trade-offs accumulate over time. A classic example of this problem is a portable application such as Mozilla using various window system toolkits; the various abstraction layers in both the application and the toolkits lead to
rather spectacularly deep call stacks with even minor exercising of functionality. While this does produce a portable application, the performance implications are significant; this tension between abstraction and implementation efficiencies forces us to reevaluate our imple-
mentations periodically. In general, layers are for cakes, not for software.

Smart Almanac

My friend Elie has written this clever on line application. Check it out: Smart Almanac is the first astrological application which computes favorable time frames for actions you may have to perform in your daily life, this is the perfect astrological assistant to organize your schedule hourly.

Quickly get a 400+ pages analysis report that will tell you hourlywhen to act in your favor, when the planets do support your action, given your natal chart, and your current location on earth.
A large choice of questions among 9 themes is available.

Bring the results with you along the the day : with the iCalendar option, synchronize with your mobile phone !

Infosec Data Analytics and Visualisation

I gave this talk at the IISYG last Friday. It was well received thanks to the reading of Raffael Marty's excellent work and blogging on loggly.

It's a nice idea but the key to success is getting application developers to adopt the logging mechanism and for team leaders to understand transactions and enforce teams to use them. There were many questions, mainly about the security of sending your logs to a third party.

Capability Operating Systems

This week Ben Laurie asks about capabilities in mainstream operating systems. Coincidentally, I was also tasked to investigate Mach Ports this week. According to the GNU Hurd wiki, "Mach Ports are Capabilities".

Okay, GNU Hurd is not exactly mainstream. But OS X is now installed in almost 11% of the computers in the US (arguably higher in more technical user populations). And Mach Ports are clearly part of OS X (including iPhone), but not widely used. Michael Weber points out that his IPC example code has found its way into the Chromium project. Maybe it's just a coincidence that Chrome was the only undefeated browser at the recent CanWest security conference in Vancouver last week.

So I am left to wonder what is the difference between these two concepts of capabilities, and what does Ben think OS X needs. Is it just a matter of making these services more visible?

Micro Trading Engines

In the search for low latency, the minimisation of hops or stack traversals is key. So here's a fairly obvious idea perhaps, but one I've yet to encounter in the real world.

If I were designing the pragmatic ideal trading engine, i.e. one which would be able to sit in a today's data center, I'd be looking an add in card-based solution with optical air inter-card interconnects - similar to the sli/crossfire approach taken to link multiple graphics card.

External interfaces would be fibre and multiple, 10GE capable at present rates so that I could accept multiple MPI IO streams for configurable bandwidth delivery.

Persistence would be staged from high speed ram through to SSD (or perhaps memristor when it hits main stream.)

Processing would be SoC with DSP, GPGPU, MMC CISC integrated with on chip with multi-gigabit optical quantum dot interconnects.

Ideally the server chassis holding the cards would be in an exchange-mart (sic) where multiple exchanges would co-locate - probably somewhere cold, geodesic with sustainable power (Iceland?)

Temporal News Signatures Used to Track Disease

Nice to see application of the techniques I described in my paper "Community-of-Interest Predicated Program Trading" used to track disease.

If they follow my logic, they'll use community expertise to do the analysis, annotation, recategorisation and dissemination to peer interest groups. Then it will be possible to build a reputation based analytics platform to summarise the trend, relate it to previous events and capture side-effects...

Open Source Intelligence in Finance

Presentation delivered to the FITE Club in 2005.

Business Driven Platform Architecture

Old presentation from 2005 where I attempted to introduce infrastructure people to the pressures the business were facing then as now...

Haskell, HPC and Crypto Get Together 22nd May 2008

Last night's get together was attended by about 15 men (women in technology take note) from a variety of backgrounds (students, cryptographers, investment bank staff.) Beers were consumed and nonsense talked - then we toddled off to Kurz and Lang for some bratwurst around eight as is our tradition (from our days in Zurich with a well known Swiss bank.)

The conclusion of the evening was that there is a place for Haskell in the city - why? Well here's two extracts from and article on Haskell and Performance on Planet Haskell written by Neil Mitchell:

Haskell's multi-threaded performance is amazing

A lot of clever people have done a lot of clever work on making multi-threaded programming in Haskell both simple and fast. While low-level speed matters for general programming, for multi-threaded programming there are lots of much higher-level performance considerations. Haskell supports better abstraction, and can better optimise at this level, outperforming C.

The trend is for higher-level optimisation

As time goes buy, higher-level programs keep getting faster and faster. The ByteString work allows programmers to write high-level programs that are competitive with C. Performance enhancements are being made to the compiler regularly, pointer tagging, constructor specialisation etc. are all helping to improve things. More long term projects such as Supero and NDP are showing some nice results. Optimisation is a difficult problem, but progress is being made, allowing programs to be written in a higher-level.

and that it would be a good idea to form a Functional Programming get together on a reasonably regular basis - so watch this space.

Financial Data Infrastructure with HDF5

Presentation given to the FITE Club in 2004.

Crisis Resilient Architecture

I gave this presentation at IISyG in 2005 to an audience of Infosec specialists. The talk was inspired by Dr Sally Leivesly of Newrisk Ltd who regularly appears on the BBC after terrorist incidents. She outlined combinatorial threat and area denial scenarios.

Another influence was the amount of fuel required to power centralised data centers which, in times of crisis, may be hard to supply regularly. Then there is the strange notion that staff will have the desire and ability to travel to some remote data center regularly which frankly beggars belief. The whole BCP/DR scenario is, sadly, driven by regulators, auditors and policy wonks.

So my take on the answer is that we need fabric/cloud computing, micro data centers and UWB MAN networks which can be up and running in an instant. Combined with the "crisis desktop", a bootable OS on a memory stick which can form a secure platform to access your virtual desktop. The parts are all there - all we need is the strategists to catch up...

Object-Orientated Security Policy

RANT Talk - Object Orientated Security Policy

I'm speaking at RANT on Object Orientated Security Policy. This is a post implementation talk which will highlight the benefits of having a visual policy framework with executable procedures arranged in an inheritance hierarchy.

The event is being described as: "Graeme’s rant will be looking into how the Security policy landscape is changing and how policies can no longer afford to be a loose collection of word documents seldom referred to. He will talk about how it is the business logic and rules of the organisation which are now becoming executable thanks to web services."

Details can be found here

Information Security - Risk Management Process

Risk assessment is a mandatory activity under all the Information Security policy frameworks e.g. COBIT, ISO27001. However, risk assessment methodologies are point-in-time and don't deal with incremental risk as the above process does.

Haskell, HPC and Crypto - Beer in the Evening

We're organising another "beer in the evening" event somewhere round about Smithfield in London to discuss Haskell, multi-core, HPC, Security and all sorts really - so if you fancy having a beer or two with a gang of gnarly, middle-aged architects, programmers and scientists (attractive huh?) then you'll be very welcome.

We will be meeting on the 22nd May - details are here

To participate, please add your name to the Doodle Poll entitled Haskell + Beer. This is so we know how many buckets of chips to order...

As All Bar One has now closed, we will be in the Bishop's Finger, Smithfield. I'm planning on getting there around 530pm. My mobile number is zero seven eight three zero three six eight zero two four - when you get there give me a call. There's a good chance we'll get some of the beers paid for too but no promises yet.

Topics for discussion/contention

• Crypto in Haskell (Dominic is Mr Haskell Crypto btw). Opportunities for parallelism (e.g.)
  
• Is Haskell and multi-core the best solution to meet the projected demand for simulation in finance or has FPGA yet to have it's day?
  
• Challenges of Haskell and high performance messaging integration. 2.4 million messages per second (not transactions) are now possible using tools like LBM from 29West.
  
• The K language (an APL derivative), Fortran, Objective-C, LISP and Smalltalk gained a small foothold in the eighties/nineties in quantitative analytics but C++ remained the main language of implementation due to availability of skillset (K programmers are like hen's teeth.) Java and now C# are now the de facto implementation languages but have serious issues for high frequency finance - will Haskell be able to make an impact or will it be another marginal language used by quants.
• Is Haskell capable of high performance?
  
• Haskell and distributed memory (memcached) performance aspects
• Haskell Data Parallelism versus state of the art FPGA approaches. Check out HPCPlatform's offering btw.
• I hear rumours that Haskell is being rolled out to the HPC team of a major investment bank in Canary Wharf - watch this space...
  

More topics very welcome.

Collaborative Web Services

Collaboration was a theme that underlay some of fundamental concepts behind the Trade Ideas platform from youDevise. This presentation highlights some of the many threads which prevailed at the time.

Open Source Intelligence in Finance

A presentation given in 2005 to the FITE Club. This was the foundations to my paper on Community-of-Interest predicated Program Trading which was presented at the Operational Research Society Conference, OR49 in September 2007.