zeroMQ - Fastest Messaging Ever
Interesting offering from zeroMQ who seem to be pals with Intel. I've checked out the code briefly and am surprised by the use of dynamic memory allocation at such a low level of granularity. I usually allocate a large lump and rely on paging to weed out the empty pages. Anyway, from the very brief look I've had, it's written in C++ and has AQMP compatibility and for some reason I get a whiff of Erlang. More analysis coming soon.
Wednesday, January 30, 2008
Tuesday, January 29, 2008
What do you mean by end-to-end latency?
Steve just sent this through:
Order execution dynamics in a global FX market
"Very nice overview. This is key: "First, you need to understand what you mean by end-to-end latency (it will be different for different types of organization)".
One of the key metrics for FX turns out to be time to cancel a trade - on EBS over 70% of $1M trades are cancelled. Nobody wants to talk about the games they are playing so it's a bit tricky to know what someone means when they say they want sub-millisecond execution."
An intuitive paper detailing the clustering, spread and timings of limit orders on EBS. It's argued that impatient traders can drop below current price for faster execution. Sitting above the price takes longer as one would imagine. There's also a correlation with order size, showing smaller volumes trade quicker. Volumes are based on 2006 data and are not particularly high.
What's missing though is the underlying need for fast price and market data. The logical implication is reduction in latency - and therefore the necessity for physical proximity. Perhaps it's time to replicate LINX idea for high performance trading applications?
Steve just sent this through:
Order execution dynamics in a global FX market
"Very nice overview. This is key: "First, you need to understand what you mean by end-to-end latency (it will be different for different types of organization)".
One of the key metrics for FX turns out to be time to cancel a trade - on EBS over 70% of $1M trades are cancelled. Nobody wants to talk about the games they are playing so it's a bit tricky to know what someone means when they say they want sub-millisecond execution."
An intuitive paper detailing the clustering, spread and timings of limit orders on EBS. It's argued that impatient traders can drop below current price for faster execution. Sitting above the price takes longer as one would imagine. There's also a correlation with order size, showing smaller volumes trade quicker. Volumes are based on 2006 data and are not particularly high.
What's missing though is the underlying need for fast price and market data. The logical implication is reduction in latency - and therefore the necessity for physical proximity. Perhaps it's time to replicate LINX idea for high performance trading applications?
Thursday, January 17, 2008
Object Orientated Security Policy
I gave a talk this week on Object Orientated Security Architecture to a Infosec management audience outside of the normal financial sectors that I work. Hopefully it was well received and made some sense. You can download the original presentation here.
Current State
I gave a talk this week on Object Orientated Security Architecture to a Infosec management audience outside of the normal financial sectors that I work. Hopefully it was well received and made some sense. You can download the original presentation here.
Current State
- Invisible policy framework
- •Loose collection of abstract policies
- •Rarely if ever read, understood or referred to
- •Complex & technical v. functional & clear
- •No business rules (BPML)
- •Monolithic & centralised v. global and federated
- •Uncertain legal status
- •Audit/accountancy driven v. business focused
- •Control rather than functionality
- •Inheritance applied to Security Policy
- •General and abstract to specific and detailed
- •Data, event and process centred
- • Detailed and technical to minimal and clear
- • Navigable framework
- •Process catalogue
- •human readable (ie not just by policy wonks)
- •machine executable
• Components
- Visual Policy framework – (BITS SIG++)
- Hierarchal process catalogue (BITS AUP++)
- •Security Architecture Capture
- subset of BITS SIG
- concentrates on data flows
- asset, data, risk classes
- business value and application complexity
- operational impact analysis
- Standardised Information Gathering (SIG) -
- Agreed Upon Procedures (AUP)
• - GLB, HIPPA, COSO, SysTrust, SOX
- Employment Law
Sunday, January 13, 2008
Haskell Cryptographic Library - New Release
Having had quite a few contributions lately, I've released a new version of the Haskell Cryptographic Library. It now contains an implementation of TEA and BubbleBabble.
Having had quite a few contributions lately, I've released a new version of the Haskell Cryptographic Library. It now contains an implementation of TEA and BubbleBabble.
Wednesday, January 02, 2008
Erlang for Five Nines Talk
Erlang for Five Nines Talk
I attended the above talk given at Skillsmatter.com on the 13th December. There were 37 attendees, two of which were women (women in technology take note.) I'm afraid I was virtually the only suit in the room apart from Francesco (the speaker) so I stuck out like the proverbial sore thumb.
It was a good talk, well received and well presented. Not particulary technical, so I'm afraid I may have asked too many questions, but I couldn't let the opportunity to get a real understanding of practical Erlang slip by. I was particularly interesting in support for multicast but as no one else had heard of it I kept quiet.
Shared Memory Support
One area I was particulary interested in was no shared memory support - all comms has to go through a tcp/ip software stack apparently. I can understand the aesthetic reasons behind this but the implications for high performance messaging are significant: from one Erlang process to another on a remote machine there's an extra two stacks to cross. It would be interesting to compare with the high performance messaging providers.
Real-world Erlang - Yaws Performance
One rather amusing piece was on the performance of Yaws, the Erlang web server, which shows very high throughput compared to Apache. Yaws manages to run 80,000 parallel sessions, serviing 2 x 20KB pages whilst poor old Apache dies after 4,000.
One unfortunate side effect of this would be to consume your organisation's entire bandwidth so I'll stick with Apache for now. If I want to ship that much (presumably) random data, the last tech I'll use is a web server. HDF5, Lustre, GFS, Gigaspaces all have a better handle on this sort of data architecture.
Conclusion
I like Erlang as a concept but they have a mountain to climb with OTP (the function library) if they're going to be more than niche. New features are being added all the time - the latest being a "posix like" threading library and support for SMP. Faster ASN.1 support too with the latest version. Erlang is growing up fast.
For those of you interested in learning Erlang, I'd recommend starting with Joe Armstrong's PhD.
I attended the above talk given at Skillsmatter.com on the 13th December. There were 37 attendees, two of which were women (women in technology take note.) I'm afraid I was virtually the only suit in the room apart from Francesco (the speaker) so I stuck out like the proverbial sore thumb.
It was a good talk, well received and well presented. Not particulary technical, so I'm afraid I may have asked too many questions, but I couldn't let the opportunity to get a real understanding of practical Erlang slip by. I was particularly interesting in support for multicast but as no one else had heard of it I kept quiet.
Shared Memory Support
One area I was particulary interested in was no shared memory support - all comms has to go through a tcp/ip software stack apparently. I can understand the aesthetic reasons behind this but the implications for high performance messaging are significant: from one Erlang process to another on a remote machine there's an extra two stacks to cross. It would be interesting to compare with the high performance messaging providers.
Real-world Erlang - Yaws Performance
One rather amusing piece was on the performance of Yaws, the Erlang web server, which shows very high throughput compared to Apache. Yaws manages to run 80,000 parallel sessions, serviing 2 x 20KB pages whilst poor old Apache dies after 4,000.
One unfortunate side effect of this would be to consume your organisation's entire bandwidth so I'll stick with Apache for now. If I want to ship that much (presumably) random data, the last tech I'll use is a web server. HDF5, Lustre, GFS, Gigaspaces all have a better handle on this sort of data architecture.
Conclusion
I like Erlang as a concept but they have a mountain to climb with OTP (the function library) if they're going to be more than niche. New features are being added all the time - the latest being a "posix like" threading library and support for SMP. Faster ASN.1 support too with the latest version. Erlang is growing up fast.
For those of you interested in learning Erlang, I'd recommend starting with Joe Armstrong's PhD.
Tuesday, December 18, 2007
Haskell - The Next Mainstream Programming Language?
Steve sent a pdf by Tim Sweeney of Epic Games entitled "The Next Mainstream Language: A Game Developers Perspective". He commented "While Sweeney doesn't exactly embrace Haskell, he seems certain that some variant will prevail".
Sweeney's 67 page presentation is a worthwhile read. It summarises the logistics of a multi-player online game - Even250K lines of C++, highly object orientated and garbage collected with associated scripts and data files. Object state evolves with gameplay. Typically there's around 1000 classes with 10K objects active during play. Events touch about 5 objects on average.
Numerical computation (for intersect detection, graph traversal etc) uses around 5 GFlops, Shading is inherently data parallel, performed on the GPU using a custom language (HLSL.) It uses around 500 GFLops
The resultant code is performance-sensitive and requires highly modular design of reliable code. Sweeney comments that C++ is poorly equiped for concurrency. This is bourne out by my own experience in the early nineties with Roguewave (remind me to tell you about the eek! bug one day.) A truism is that analysing performance isn't a simple matter of identifying "hot spots" and optimising that code - or writing assembly language to get around an issue. This is never done as Sweeney notes. Event timing and handling, data structure construction, execution path may have to be revisitied resulting in a lack of productivity. A the end of the day - you have to compromise performance vs productivity.
Sweeney then proceeds to deconstruct C++ as a language and Object-Orientation as a principle. As you've probably found yourself, sometimes the base class has to change - leading to significant refactoring. In particular, he slates the inability of C++ compiler to handle dynamic failure, contrasting how Haskell handles this with ease. C# also comes in for a wry observation: it exposes 10 types of integer-like data types, none of which are those defined by Pythagoras...
Pushing Haskell, he hypothesises that 80% of CPU effort could be parallelised using Haskel ST - the Strict State Thread module and, further still, that programmer "hints" can be used to guide parallelisation - stating that Haskell's Monadic nature is a natural fit for parallelisation - imperative programming is perhaps the "wrong trousers".
As I found to my cost, "shared concurrency is [indeed] hopelessly intractible" in my "Why events are a bad idea" post. I too got in a mess with manual synchronisation. Sweeney also suggests as the quoted paper does, that message passing is the recipe for high performance because, when combined with software tranactional memory and multiple threads, the overhead of synchronisation is bearable when the object population his rate is small (~ 5 per 10000 per message.)
The article then concludes with his musings on the next generation language. All-in-all, a nice empirical paper from the coal face.
Next, I'll cast my eye on Erlang - there's buzz there too from the finance crowd, but I know what my next language is going to be - and hit begins with H...
Sweeney's 67 page presentation is a worthwhile read. It summarises the logistics of a multi-player online game - Even250K lines of C++, highly object orientated and garbage collected with associated scripts and data files. Object state evolves with gameplay. Typically there's around 1000 classes with 10K objects active during play. Events touch about 5 objects on average.
Numerical computation (for intersect detection, graph traversal etc) uses around 5 GFlops, Shading is inherently data parallel, performed on the GPU using a custom language (HLSL.) It uses around 500 GFLops
The resultant code is performance-sensitive and requires highly modular design of reliable code. Sweeney comments that C++ is poorly equiped for concurrency. This is bourne out by my own experience in the early nineties with Roguewave (remind me to tell you about the eek! bug one day.) A truism is that analysing performance isn't a simple matter of identifying "hot spots" and optimising that code - or writing assembly language to get around an issue. This is never done as Sweeney notes. Event timing and handling, data structure construction, execution path may have to be revisitied resulting in a lack of productivity. A the end of the day - you have to compromise performance vs productivity.
Sweeney then proceeds to deconstruct C++ as a language and Object-Orientation as a principle. As you've probably found yourself, sometimes the base class has to change - leading to significant refactoring. In particular, he slates the inability of C++ compiler to handle dynamic failure, contrasting how Haskell handles this with ease. C# also comes in for a wry observation: it exposes 10 types of integer-like data types, none of which are those defined by Pythagoras...
Pushing Haskell, he hypothesises that 80% of CPU effort could be parallelised using Haskel ST - the Strict State Thread module and, further still, that programmer "hints" can be used to guide parallelisation - stating that Haskell's Monadic nature is a natural fit for parallelisation - imperative programming is perhaps the "wrong trousers".
As I found to my cost, "shared concurrency is [indeed] hopelessly intractible" in my "Why events are a bad idea" post. I too got in a mess with manual synchronisation. Sweeney also suggests as the quoted paper does, that message passing is the recipe for high performance because, when combined with software tranactional memory and multiple threads, the overhead of synchronisation is bearable when the object population his rate is small (~ 5 per 10000 per message.)
The article then concludes with his musings on the next generation language. All-in-all, a nice empirical paper from the coal face.
Next, I'll cast my eye on Erlang - there's buzz there too from the finance crowd, but I know what my next language is going to be - and hit begins with H...
Monday, December 17, 2007
Applied Infoviz and Knowledge Re-injection
The Infoviz toolkit is used at Project Seven by a friend of mine et al. They're working on an intelligence analysis tool which supports reinjection of explicit knowledge earlier in the categorisation/discovery chain to guide discovery.
This is a different approach to the one I took in my signature based approach in my paper "Community-of-interest predicated program trading" where I suggested using centroid categorisation augmented with off-center categories. The Project Seven techinque relies on heuristically directed iteration (or as it's known in lay terms - trial and error) where my technique relies on the visualisation of the categorisation centroid. I think both approaches have merit and would produce good results.
This is a different approach to the one I took in my signature based approach in my paper "Community-of-interest predicated program trading" where I suggested using centroid categorisation augmented with off-center categories. The Project Seven techinque relies on heuristically directed iteration (or as it's known in lay terms - trial and error) where my technique relies on the visualisation of the categorisation centroid. I think both approaches have merit and would produce good results.
Friday, December 07, 2007
Fork/Join - The End of the Road for FPGA/GPU/ASIC Gang?
Steve W sent this article from IBM - Java Theory and Practise: Stick a fork in it
"One of the additions to the
Steve says "Looks like java 7 will have built-in support for the Fork/Join pattern (i.e. similar to what Erlang has and what the Squeak guys have been talking about)"
Is this the end for the short-lived FPGA/GPU bandwagon? It could well be - with massively multi-core chips round the corner, they are considerably more attractive an option with their on core FPU and low-latency cache etc. Solves a whole lot of issues around buying and supporting esoteric hardware it would seem.
"One of the additions to the
java.util.concurrent packages coming in Java™ 7 is a framework for fork-join style parallel decomposition. The fork-join abstraction provides a natural mechanism for decomposing many algorithms to effectively exploit hardware parallelism."Steve says "Looks like java 7 will have built-in support for the Fork/Join pattern (i.e. similar to what Erlang has and what the Squeak guys have been talking about)"
Is this the end for the short-lived FPGA/GPU bandwagon? It could well be - with massively multi-core chips round the corner, they are considerably more attractive an option with their on core FPU and low-latency cache etc. Solves a whole lot of issues around buying and supporting esoteric hardware it would seem.
Monday, December 03, 2007
Talk: Erlang for Five Nines by Francesco Cesarini
Looks like an opportunity for a beer afterwards?
"Francesco Cesarini will be presenting the concurrent soft real time functional programming language Erlang and its rapidly growing community. He will describe why Erlang programs are generally 4 - 10 times shorter than their counterparts in Java, C and C++ achieving 99,999% availability. Ongoing research projects in the UK and overseas which will be covered, including refactoring, AI, type systems and control systems for robots. The talk will also go into why Erlang, originally invented to handle the next generation of Telecom products, has been successful in a much wider range of sectors including banking and e-commerce. "
Date: Thursday, 13th December 2007
Time:18:30-20:30
Venue:
Skills Matter
1 Sekforde Street
London EC1R 0BE
Near Farringdon Rail/Tube
Registration http://skillsmatter.com/menu/85
Looks like an opportunity for a beer afterwards?
"Francesco Cesarini will be presenting the concurrent soft real time functional programming language Erlang and its rapidly growing community. He will describe why Erlang programs are generally 4 - 10 times shorter than their counterparts in Java, C and C++ achieving 99,999% availability. Ongoing research projects in the UK and overseas which will be covered, including refactoring, AI, type systems and control systems for robots. The talk will also go into why Erlang, originally invented to handle the next generation of Telecom products, has been successful in a much wider range of sectors including banking and e-commerce. "
Date: Thursday, 13th December 2007
Time:18:30-20:30
Venue:
Skills Matter
1 Sekforde Street
London EC1R 0BE
Near Farringdon Rail/Tube
Registration http://skillsmatter.com/menu/85
Thursday, November 29, 2007
One Thread Per Core
Way back in 1990, when I was performance tuning Ingres on a multi-processor Sequent machine, I found that using processor affinity - ie tieing the main process to a particular CPU gave the best performance. Intuitively I figured that this was due to not having to context switch but never had the time to prove my theory.
More recently, numa issues have been surfacing again in the search for low latency.
A friend of mine has recenly been looking at k-way sorting and the thread scheduling again has come into play. I have always been curious about the analysis of Numa architecture but never got to spend that much time on it. In the old days of single core, context would have to be shared over the bus, Numa was a great advance, introducing a high performance inter-cpu/die channel with about 6GB/s throughput. Now with multicore it's all on the die but appears to be still an issue as the following results show from a friend show:
The k-way merge sort looks pretty good, almost twice as fast as a mono-threaded quicksort. Here some results for 10 millions numeric (double) values to sort :
sort 1st part done (parallel segments) in 3672ms
sort 2nd part done in 3157ms niter=19999993
Total time for k-way merge sorting 10000000 double values=6922 ms
Total time for monoquicksorting 10000000 double values=13609 ms
Now if you look at the performance variation when sorting 2 millions items, it appears that the best performance occurs when the number of parallel sorting threads equals the number of core available from the CPU (my box is a mono quad-core Xeon 5355 2.6Ghz) :
2 threads used:
sort 1st part done (parallel segments) in 1031ms
sort 2nd part done in 469ms niter=1999999
Total time for k-way merge sorting 2000000 double values=1562 ms
Total time for monoquicksorting 2000000 double values=1984 ms
3 threads used :
sort 1st part done (parallel segments) in 688ms
sort 2nd part done in 563ms niter=3333329
Total time for k-way merge sorting 2000000 double values=1281 ms
Total time for monoquicksorting 2000000 double values=2015 ms
4 threads used :
sort 1st part done (parallel segments) in 516ms
sort 2nd part done in 594ms niter=3999995
Total time for k-way merge sorting 2000000 double values=1125 ms
Total time for monoquicksorting 2000000 double values=2000 ms
8 threads used :
sort 1st part done (parallel segments) in 437ms
sort 2nd part done in 781ms niter=5999991
Total time for k-way merge sorting 2000000 double values=1234 ms
Total time for monoquicksorting 2000000 double values=2047 ms
16 threads used :
sort 1st part done (parallel segments) in 391ms
sort 2nd part done in 906ms niter=7999975
Total time for k-way merge sorting 2000000 double values=1313 ms
Total time for monoquicksorting 2000000 double values=2031 ms
More recently, numa issues have been surfacing again in the search for low latency.
A friend of mine has recenly been looking at k-way sorting and the thread scheduling again has come into play. I have always been curious about the analysis of Numa architecture but never got to spend that much time on it. In the old days of single core, context would have to be shared over the bus, Numa was a great advance, introducing a high performance inter-cpu/die channel with about 6GB/s throughput. Now with multicore it's all on the die but appears to be still an issue as the following results show from a friend show:
The k-way merge sort looks pretty good, almost twice as fast as a mono-threaded quicksort. Here some results for 10 millions numeric (double) values to sort :
sort 1st part done (parallel segments) in 3672ms
sort 2nd part done in 3157ms niter=19999993
Total time for k-way merge sorting 10000000 double values=6922 ms
Total time for monoquicksorting 10000000 double values=13609 ms
Now if you look at the performance variation when sorting 2 millions items, it appears that the best performance occurs when the number of parallel sorting threads equals the number of core available from the CPU (my box is a mono quad-core Xeon 5355 2.6Ghz) :
2 threads used:
sort 1st part done (parallel segments) in 1031ms
sort 2nd part done in 469ms niter=1999999
Total time for k-way merge sorting 2000000 double values=1562 ms
Total time for monoquicksorting 2000000 double values=1984 ms
3 threads used :
sort 1st part done (parallel segments) in 688ms
sort 2nd part done in 563ms niter=3333329
Total time for k-way merge sorting 2000000 double values=1281 ms
Total time for monoquicksorting 2000000 double values=2015 ms
4 threads used :
sort 1st part done (parallel segments) in 516ms
sort 2nd part done in 594ms niter=3999995
Total time for k-way merge sorting 2000000 double values=1125 ms
Total time for monoquicksorting 2000000 double values=2000 ms
8 threads used :
sort 1st part done (parallel segments) in 437ms
sort 2nd part done in 781ms niter=5999991
Total time for k-way merge sorting 2000000 double values=1234 ms
Total time for monoquicksorting 2000000 double values=2047 ms
16 threads used :
sort 1st part done (parallel segments) in 391ms
sort 2nd part done in 906ms niter=7999975
Total time for k-way merge sorting 2000000 double values=1313 ms
Total time for monoquicksorting 2000000 double values=2031 ms
Thursday, November 08, 2007
LSE Goes Boom!
"No other exchange is undertaking such an ambitious technology refresh programme based on next-generation Microsoft technology. We have always provided a first-class service, but now we can claim to be the fastest in the world as well."
- David Lester, CIO, London Stock Exchange
Now I wonder why no other exchange uses Microsoft technology for high performance messaging then...
The LSE was a big coup for Microsoft - one of the worlds leading exchanges which, until the Infolect platform based on .Net technology came along, had no outages since 1987. Even more embarassing are the anti-linux rhetoric contained in the "Get The Facts" campaign quoting the above David Lester quote - a nice analysis of which you can find here from Paul Murphy. He concludes that the so called high performance message delivery is the same as a "relatively slow LAN".
This won't be the last time this occurs - in my operational experience, solutions based on .Net exhibited mysterious freezes which were never solved.
"No other exchange is undertaking such an ambitious technology refresh programme based on next-generation Microsoft technology. We have always provided a first-class service, but now we can claim to be the fastest in the world as well."
- David Lester, CIO, London Stock Exchange
Now I wonder why no other exchange uses Microsoft technology for high performance messaging then...
The LSE was a big coup for Microsoft - one of the worlds leading exchanges which, until the Infolect platform based on .Net technology came along, had no outages since 1987. Even more embarassing are the anti-linux rhetoric contained in the "Get The Facts" campaign quoting the above David Lester quote - a nice analysis of which you can find here from Paul Murphy. He concludes that the so called high performance message delivery is the same as a "relatively slow LAN".
This won't be the last time this occurs - in my operational experience, solutions based on .Net exhibited mysterious freezes which were never solved.
The Transition from N-Tier to High Performance Architecture
This is a little story of Anybank, who have a classic three-tier architecture using in-house developed solutions. They are embarking on a plan to improve the architecture, aiming for scalability, globalisation and integration in order to meet predicted growth in volumes.
In order to achieve this, the architecture will be transitioned from a transaction-orientated platform to an event-driven, peer-to-peer, asynchronous, persistent messaging-based architecture. Anybank have recognised that in order to achieve this, they must move away from in-house developed solutions and take advantage of open source and third-party products.
Architectural Components
Anybank have a classic three-tier architecture with a moderate level of fan out/in.
At the front end, there are a series of internal trading and control applications and several external client applications and gateways to systems such as FIX.
The communication from tier 3 to tier 2 is TCP based, carrying FIX or HTTP traffic to series of middle tier servers. Additionally, there is a custom protocol used by the thick client to talk to the middle tier servers. This is an efficient ASCII based protocol with pipe-delimited fields.
External Applications
The system has FIX connections to several exchanges and markets. Clients have a choice of a thick client or a web front end. Quotes are sent over FIX at a rate of 4/second.
Internal Applications and Services
There are several internal applications that are used by trading, credit and risk control and operations. The connections to these are managed by a series of front office servers and talk to the database tier for information. They run a generic set of applications that would be found in any other bank:
Anybank Tiered Architecture Analysis
Several key components have been targeted in an attempt to reduce potential bottlenecks in the system. Serial activities are being replaced with parallel activities, single threaded code is being rewritten to be multhreaded.
However, by applying HPC techniques and chosing the right products, we can gain significant performance quickly and robustly.
Tier 1 - Database Persistence Layer
This is the database layer, which is currently a replicated SQL Server. As with all relational databases, the maximal transaction rate is a function of the IO subsystem, log file transaction rate, number of indexes, data volumes and access patterns. Typically, one can expect a transaction rate of between 1-5 thousand transactions per second, which translates into 5-1 milliseconds per transaction.
Business level transactions may consist of many database transactions, which results in the potential for adding considerable latency to client and API calls.
Tier 2 - Application Services and Access Control Layer
The application logic layers combines authentication, access control and authorisation with business logic, static and derived data, streaming market data and database transactions. It is traditionally at this layer where fan-in/out is facilitated.
Tier 3 - Application Layer
The application layer consists of traditional thick, web and FIX clients, each with their own particular data access profile. Thick clients typically are used when streaming data is consumed by, for example, a pricing application. They may consume data delineated by topic or routed by function.
Web clients typically consume http/https/javascript and/or xml data to drive form-based, browser applications.
FIX clients can potentially deliver 100K messages per second when using FAST, the FIX compression protocol.
HPC Tools Applied to Anybank
The architecture is in transition from client server to an event-based and message orientated structure in order to increase resilience and enable scalability. The architecture has already some elements of fan out via multiple services connected via TCP.
There are opportunities to augment the current architecture and improve transaction capacity whilst reducing latency in each of the three tiers of the current Anybank architecture using Open Source messaging and caching products which have proven robustness in real-world applications.
Tier 1 and 2 - Peer-to-peer Message Bus
The publish and subscribe paradigm is ideal to use where multiple senders and receivers or combinations of the two are present and is common pattern for trading applications where prices, curves, derived data, immediate messages and topic based mechanisms need to be catered for.
In Tier 1 and 2, it is proposed to use the MantaRay peer-to-peer messaging product to provide message caching to tier 2 applications and services, effectively decoupling tier 1 and 2. MantaRay Messaging persistence logic guarantees delivery to unavailable peers once they come back online. Sending nodes need not await responses from a broker before resuming
other tasks. Such capabilities are critical issues in high availability environments. This is considerably faster than a synchronous database access thereby accelerating application performance.
Also In tier 2, application/service fan out will be added using the pub/sub capabilities of the MantaRay solution.
Reliable Multicast over the WAN
MantaRay also has a WAN Bridge capability which accepts connections from other MantaRay peers thereby getting around multi-cast over the WAN issues. This would allow the mixing of the benefits of multicast with the recoverability of MantaRay.
This could be used to distribute real-time market data globally. Peer-to-peer is ideal for this purpose. It can be configured to deliver topic orientated messages over multiple multicast streams, layering the topics in order to use bandwidth effectively using source paced delivery which would be ideal for Anybank.
to be continued...
In order to achieve this, the architecture will be transitioned from a transaction-orientated platform to an event-driven, peer-to-peer, asynchronous, persistent messaging-based architecture. Anybank have recognised that in order to achieve this, they must move away from in-house developed solutions and take advantage of open source and third-party products.
Architectural Components
Anybank have a classic three-tier architecture with a moderate level of fan out/in.
At the front end, there are a series of internal trading and control applications and several external client applications and gateways to systems such as FIX.
The communication from tier 3 to tier 2 is TCP based, carrying FIX or HTTP traffic to series of middle tier servers. Additionally, there is a custom protocol used by the thick client to talk to the middle tier servers. This is an efficient ASCII based protocol with pipe-delimited fields.
External Applications
The system has FIX connections to several exchanges and markets. Clients have a choice of a thick client or a web front end. Quotes are sent over FIX at a rate of 4/second.
Internal Applications and Services
There are several internal applications that are used by trading, credit and risk control and operations. The connections to these are managed by a series of front office servers and talk to the database tier for information. They run a generic set of applications that would be found in any other bank:
- Market Making, Trading Station and Sales Trader Workbench: Prop. trading tools
- Session Management: Routing, STP gateways, reception of execution reports.
- Order Moniting: provides snapshot requests and streaming prices
- Client Margin Calculation Engine: Limit checking, credit check, event publishing service
- Exposure and Barrier Monitoring
- Front Office Server: Authentication, Authorisation and Access control (AAA), Event notification.
- Trade Server: Visual trading, large trades and limit trading.
- Market Data Servers: Takes streaming market prices from feed handlers and distributes Market Data
- Metrics and SLA monitoring
Anybank Tiered Architecture Analysis
Several key components have been targeted in an attempt to reduce potential bottlenecks in the system. Serial activities are being replaced with parallel activities, single threaded code is being rewritten to be multhreaded.
However, by applying HPC techniques and chosing the right products, we can gain significant performance quickly and robustly.
Tier 1 - Database Persistence Layer
This is the database layer, which is currently a replicated SQL Server. As with all relational databases, the maximal transaction rate is a function of the IO subsystem, log file transaction rate, number of indexes, data volumes and access patterns. Typically, one can expect a transaction rate of between 1-5 thousand transactions per second, which translates into 5-1 milliseconds per transaction.
Business level transactions may consist of many database transactions, which results in the potential for adding considerable latency to client and API calls.
Tier 2 - Application Services and Access Control Layer
The application logic layers combines authentication, access control and authorisation with business logic, static and derived data, streaming market data and database transactions. It is traditionally at this layer where fan-in/out is facilitated.
Tier 3 - Application Layer
The application layer consists of traditional thick, web and FIX clients, each with their own particular data access profile. Thick clients typically are used when streaming data is consumed by, for example, a pricing application. They may consume data delineated by topic or routed by function.
Web clients typically consume http/https/javascript and/or xml data to drive form-based, browser applications.
FIX clients can potentially deliver 100K messages per second when using FAST, the FIX compression protocol.
HPC Tools Applied to Anybank
The architecture is in transition from client server to an event-based and message orientated structure in order to increase resilience and enable scalability. The architecture has already some elements of fan out via multiple services connected via TCP.
There are opportunities to augment the current architecture and improve transaction capacity whilst reducing latency in each of the three tiers of the current Anybank architecture using Open Source messaging and caching products which have proven robustness in real-world applications.
Tier 1 and 2 - Peer-to-peer Message Bus
The publish and subscribe paradigm is ideal to use where multiple senders and receivers or combinations of the two are present and is common pattern for trading applications where prices, curves, derived data, immediate messages and topic based mechanisms need to be catered for.
In Tier 1 and 2, it is proposed to use the MantaRay peer-to-peer messaging product to provide message caching to tier 2 applications and services, effectively decoupling tier 1 and 2. MantaRay Messaging persistence logic guarantees delivery to unavailable peers once they come back online. Sending nodes need not await responses from a broker before resuming
other tasks. Such capabilities are critical issues in high availability environments. This is considerably faster than a synchronous database access thereby accelerating application performance.
Also In tier 2, application/service fan out will be added using the pub/sub capabilities of the MantaRay solution.
Reliable Multicast over the WAN
MantaRay also has a WAN Bridge capability which accepts connections from other MantaRay peers thereby getting around multi-cast over the WAN issues. This would allow the mixing of the benefits of multicast with the recoverability of MantaRay.
This could be used to distribute real-time market data globally. Peer-to-peer is ideal for this purpose. It can be configured to deliver topic orientated messages over multiple multicast streams, layering the topics in order to use bandwidth effectively using source paced delivery which would be ideal for Anybank.
to be continued...
Wednesday, October 31, 2007
Merchant Bankers finally get the Internet and FIX?
From second Tanenbaum elephant hump comes the story about how emerging markets are booming because of gasp! - the application of internet technology to investment banking. It's rather ironic that the technology was there in 1970 - trouble was - the business just wasn't listening - messaging is as old as the hills - FIX is messaging in verbose clothing (XML) - suitable for the trading of web "services" by the externalisation of business processes as I observed in my 1999 paper on "The Role of XML in Enabling Business Solutions Built from Collaborative Web-Based Services". This was further extended in the blog article here.
Monday, October 29, 2007
Pot, Kettle, Black
I had to laugh last week, when I had dinner conversation with an IT journalist. Some fairly innocent conversation about firearms, internet security and shredding documents came up. But whilst the journalist writes stirring columns about ecommerce; security; credit card fraud and atms, he blanked me when asked about identity theft and cross-shredding shredders. Of course he has home insurance, and expects some sort of burgalar. But stealing credit card statements and gas bills. Thats the assumption that his new live-in girlfriend wont read through the pile, and decide what he can afford or not for xmas. So where does this guy keep his password or atm pin. Oh how silly of me, its in his wallet of course. (that I dared not to ask)
I had to laugh last week, when I had dinner conversation with an IT journalist. Some fairly innocent conversation about firearms, internet security and shredding documents came up. But whilst the journalist writes stirring columns about ecommerce; security; credit card fraud and atms, he blanked me when asked about identity theft and cross-shredding shredders. Of course he has home insurance, and expects some sort of burgalar. But stealing credit card statements and gas bills. Thats the assumption that his new live-in girlfriend wont read through the pile, and decide what he can afford or not for xmas. So where does this guy keep his password or atm pin. Oh how silly of me, its in his wallet of course. (that I dared not to ask)
Monday, September 03, 2007
IP and the Internet
A friend of mine, Dr Tim Sampson is running an event on the above subject. Areas he will cover are:
A friend of mine, Dr Tim Sampson is running an event on the above subject. Areas he will cover are:
- Domain name rights and linking
- Copyright, database rights, framing and “borrowed content”;
- The Use and Abuse of Branding on Websites and Trademark and brand issues;
- Advertising via the Web – Its Limits and Liabilities;
- Digital Downloads – What you can and cannot do;
- Streaming, Webcasting and Internet TV
- Patents and Designs
- The Trade in Regulated Goods
- Commercial Exploitation of Database Rights over the Web
Wednesday, August 29, 2007
Beer in the Evening - Data Parallelisation and Functional Languages in Finance - September 25th 2007
It's been suggested that we get together again for another beer in the evening by John Barr who has recently joined the 451 Group from Intel. John wrote an article called "Reinventing the future: parallelism and acceleration" which tracked the recent rise in accelerated computing and is a comprehensive review of the area.
I'll put up some reading material in due course. Watch this space.
So if you're up for a beer and a casual chat about HPC, HFF, low latency, parallelism, messaging vs events etc we'll be meeting in the Red Lion, 8 Lombard Court, London from 630pm onwards on the 25th September.
You can call me or email on 0791 505 5380 or rgb at enhyper.com if you have any questions.
It's been suggested that we get together again for another beer in the evening by John Barr who has recently joined the 451 Group from Intel. John wrote an article called "Reinventing the future: parallelism and acceleration" which tracked the recent rise in accelerated computing and is a comprehensive review of the area.
I'll put up some reading material in due course. Watch this space.
So if you're up for a beer and a casual chat about HPC, HFF, low latency, parallelism, messaging vs events etc we'll be meeting in the Red Lion, 8 Lombard Court, London from 630pm onwards on the 25th September.
You can call me or email on 0791 505 5380 or rgb at enhyper.com if you have any questions.
Micro and Macro Performance Analytics - a Log with Two Tales
Micro and Macro Performance Analytics - a Log with Two Tales
Sometime when we measure things we end up introducing latency into the system. I recently came across a neat scheme where a timing message was “stamped” on every hop then rerouted to the source. The average round trip time was calculated by dividing the elapsed time in two. The overall times produced were adequate for what the guy required but there was an annoying latency spike which happened periodically and was a cause for concern. The average trip time from source to receiver was 900 microseconds. Occasionally, a 9 millisecond time was reported which, if echoed in production, would have been unacceptable. To find the problem, all the messages above 1ms were tagged and grep’d out into a file. Using vi to analyse this log file, it was noted that the latency spikes had a definite periodicity of around 600ms.
Visual inspection is a great technique as the brain is good at recognising patterns. However, as we can now generated millions of log entries per second, this tactic has severe limitations in that it highlights micro trends, therefore It’s important to visualise the data too so that macro trends can be recognised. Luckily I did as there was one lurking in the data. I took a minute’s worth of >1ms log file entries and manipulated them in good old excel to produce the graph below:
As you see from the graph, there's a an obvious 10 second event which is highly regular in it's form. On further investigation, it turned out to be a call to a summarisation routine which caused the spikes to cease momentarily. Now it's lucky this was found using a synthetic load - in production, it could be a lot more difficult to spot. On the plus side, it's nice to come across a well designed load simulation - this is seldom the case. People are prone to quote averages and pay little attention to the spikes - but in high frequency trading, it's the spike that will lose you big money.
So, be careful what you measure and be careful that by measuring, you don't introduce side effects which could so easily be hidden in production.
Sometime when we measure things we end up introducing latency into the system. I recently came across a neat scheme where a timing message was “stamped” on every hop then rerouted to the source. The average round trip time was calculated by dividing the elapsed time in two. The overall times produced were adequate for what the guy required but there was an annoying latency spike which happened periodically and was a cause for concern. The average trip time from source to receiver was 900 microseconds. Occasionally, a 9 millisecond time was reported which, if echoed in production, would have been unacceptable. To find the problem, all the messages above 1ms were tagged and grep’d out into a file. Using vi to analyse this log file, it was noted that the latency spikes had a definite periodicity of around 600ms.
Visual inspection is a great technique as the brain is good at recognising patterns. However, as we can now generated millions of log entries per second, this tactic has severe limitations in that it highlights micro trends, therefore It’s important to visualise the data too so that macro trends can be recognised. Luckily I did as there was one lurking in the data. I took a minute’s worth of >1ms log file entries and manipulated them in good old excel to produce the graph below:
As you see from the graph, there's a an obvious 10 second event which is highly regular in it's form. On further investigation, it turned out to be a call to a summarisation routine which caused the spikes to cease momentarily. Now it's lucky this was found using a synthetic load - in production, it could be a lot more difficult to spot. On the plus side, it's nice to come across a well designed load simulation - this is seldom the case. People are prone to quote averages and pay little attention to the spikes - but in high frequency trading, it's the spike that will lose you big money.So, be careful what you measure and be careful that by measuring, you don't introduce side effects which could so easily be hidden in production.
Wednesday, August 01, 2007
Community of Interest Predicated Program Trading
I've had my paper on the above reviewed and accepted for publication and presentation to the Knowledge Management stream of this year's Operational Research Society conference, OR49 in Edinburgh. I'll be attending the conference for the three days and hanging out with Dr Duncan Shaw, an expert in Soft Systems Methodologies at Aston University.
You can download the paper from the Enhyper site here.
I've had my paper on the above reviewed and accepted for publication and presentation to the Knowledge Management stream of this year's Operational Research Society conference, OR49 in Edinburgh. I'll be attending the conference for the three days and hanging out with Dr Duncan Shaw, an expert in Soft Systems Methodologies at Aston University.
You can download the paper from the Enhyper site here.
Friday, July 20, 2007
A Design Pattern Template for High Performance Messaging
This is a work in progress. High performance messaging is hard. There are many parameters which can be configured (or misconfigured) so I'm working on a series of pattern templates. Work in progress...
Pattern Name
{ Source: Single.All Topics, Receiver: Multiple.All Topics }
Synonyms
Related Patterns
{ Source: Per Topic, Receiver: Per Topic }
Overview
Topic Methodology
Source per topic
{ Single.Source , Receiver.All Topics }
Group Rate Control Policy
Extreme 1 { Sender.Rate.Slowest Receiver, Reciever:at leisure)
Extreme 2 { Sender: Fast As Possible, Receiver: Try to keep up)
Middle Ground (Send.Minimum Data Loss, Receiver.Minimise Latency)
Transport Protocol
Example
Diagram
Operating Environment
Related Patterns
This is a work in progress. High performance messaging is hard. There are many parameters which can be configured (or misconfigured) so I'm working on a series of pattern templates. Work in progress...
Pattern Name
{ Source: Single.All Topics, Receiver: Multiple.All Topics }
Synonyms
Related Patterns
{ Source: Per Topic, Receiver: Per Topic }
Overview
Topic Methodology
Source per topic
{ Single.Source , Receiver.All Topics }
Group Rate Control Policy
Extreme 1 { Sender.Rate.Slowest Receiver, Reciever:at leisure)
Extreme 2 { Sender: Fast As Possible, Receiver: Try to keep up)
Middle Ground (Send.Minimum Data Loss, Receiver.Minimise Latency)
Transport Protocol
Example
Diagram
Operating Environment
Related Patterns
So How Fast is an APR Hash Table?
Just ran a quick metric on uploading approx 20K stock information items into an APR hash table on a heavily loaded (load avg ~4), Sparc III based Solaris 8 machine. It took 32ms to load which works out roughly 1 microsec per apr_hash_set() call. Each data item was roughly 140 bytes long - not bad for an old machine.
Just ran a quick metric on uploading approx 20K stock information items into an APR hash table on a heavily loaded (load avg ~4), Sparc III based Solaris 8 machine. It took 32ms to load which works out roughly 1 microsec per apr_hash_set() call. Each data item was roughly 140 bytes long - not bad for an old machine.
Subscribe to:
Posts (Atom)
View Graeme Burnett's profile