News Analysis for Program Trading

As previously posted, I'm writing a paper on news based program trading for the KM stream at the Operational Research Society's Annual Conference in September. This paper is the culmination of many years research and interest in the area of new analysis and I hope to show that the application of statistical techniques combined with visualisation can lead to an effective intelligence system which solves some of the conundrums facing traders, and for that matter, intelligence analysts.

The goal is to greatly shorten the time to disseminate events to the people who need to consume them, allowing them to act on this information. However, there's also an intention to analyse the likely outcome of this interaction and put in place a strategy to take advantage of this event. Another hypothetical outcome is that event "signatures" will be recognised and effects correlated in different sectors.

News Analysis

The first goal is to simplify the elements of news which we will analyse. To do this, I propose to model the way that people tend to read newspapers and select stories which interest them. Some read from front to back, others select favourite sections first, others, and I include myself here, read from back to front.

When we read, the first element to be considered is either the title or a picture. The writer of the article has to aphoristically state the contents of the news in an attempt to get the readers interest.
The title also contains other information like people, places, sectors, amounts, therefore this is the key piece which is used for presentation to the end user.

The rest of the story consists of a series of sentences arranged into paragraphs. Within the story will also be the information we are interested in. The relativity of people can be used to build a Social Network Analysis graph based on proximity. If two people are mentioned in the same sector (e.g. FX trading) they are related. If they are mentioned in the same publication they are related more closely. The same story, closer still. Same paragraph, even closer. Same sentence, the closest. From this we can draw a graph showing the individuals "social network". There's a very good example of this at www.namebase.org where you can perform useful searches on people involved in the intelligence world from their appearance in related publications.

Topographical Mapping

News also contains physical places. Mapping individuals, companies, sectors, amounts to physical location can reveal useful information and is a technique much used in policing.

Categorisation

Categorisation is something humans do every day and is fundamental to our heuristic judgement. Humans are very good at it, however, what they're not so good at is dealing with something which falls into multiple categories.


To be continued shortly...