Logging Sins, Hierarchical Tuples and Effective Security Architecture

Logging Sins, Hierarchical Tuples and Effective Security Architecture

There are many sins in logging, none so amusing than this one I saw recently:

+++ ALERT : SEVERE : Invalid Market Order Submission ... Hence, releasing the Trader Order in Panic... +++ALERT : SEVERE+++

In the old days, we used to return a number which was then looked up in a decode table. This lead to obscure code and errors which were quite severe when the DBA messed up the decode table...

However, as an aid to debugging, self-describing code can be useful. So if you're tempted, try tuples - they're quite neat. They're a feature of functional languages, in particular Haskell. Here are some real examples:

TIB.SEND.TOALGO.AMENDTRADERORDER
TIB.SEND.TOALGO.ORDER.CANCEL
LBM.CALLBACK.INVOKE
LBM.CALLBACK.RETURN

Without commentary, you can deduce what's happening with this code and these messages. Tuples are cool. They can also be used to describe design patterns. I wrote a paper a while back which I presented at the IISyG on Security Architecture Patterns - I'll post the deck soon on the enhyper subversion server. I took an idea from the Antipatterns book on naming of "antipatterns" and crystallised the methodology to this tuple for describing a pattern:

{GENERAL, LESS GENERAL, SPECIFIC, MORE SPECIFIC}

Applied to the concept of security, this tuple looks like this:

Generic Security Concept.Application Context.Application Instance.Configuration

So I came up with patterns which looked like this:

DataTransfer.BBCP
SecrecyJurisdiction.Authentication.SSH.PasswordlessCertificate
Authentication.Applicaiton.JavaClient.OracleOID

Now the beauty of this scheme is that people are lead to logically to the comprehension of the solution by hierarchical decomposition. Add the ability to have arbitrary tuple lengths and encourage stereotypes and you now have well known patterns which you can back by implementations. Now here's the killer part - get security risk to certify/risk assess the implementation so you now have effective security in a risk compliant manner.

Unfortunately, this bright idea foundered for two reason - banks are here to make money, not develop software and security risk guys found it difficult to think like developers.

Local Disk Hotspot Analysis

One common mistake I see is NFS/Samba mounted partitions use to either stage data into memory or to write application log files to. There's no real excuse for this, and the argument that local storage is not allowed by intrastructure engineering policy is one which I easily argued against at a tier 1. We wanted about half a gig of file systems space to store logs and model data which could be loaded quickly in the event of a system or application crash. We were told that this was not allowed under any circumstances as it was against policy.

The issue we had was quite simple - on our Solaris box, restarting the application took 2 hours, pulling the data from an an ancient disk array with really poor service times and some config stuff NFS mounted partition. When we moved to Linux and used our local home dir to stage the models and store the logs, this went to 15 mins - an acceptable restart time for the business. So I arranged a meeting with the Head of Unix Infrastructure and told him that he needed to explain to the business why they couldn't trade for one and three quarter hours in the event of a system failure. For some reason he changed his mind and we managed to get two fast SATA II hard drives in addition to the two OS drives.

JBOD Arrays

If you are using a JBOD array with some form of disk suite management, frequently there's several performance problems that go unnoticed and unchecked - hardly surprising knowing the extreme workloads of some to the sys admins I've worked with. This leaves them no time to be proactive.

The first is what I call disk hot-spotting - where a bunch of disks are idle yet one is maxed due to poor partitioning. To analyse this, I wrote diskanalyse which aggregates sar output to highlight possible issue. Here's an example of two disks which just so happen to be the OS disks. It shows that there's a poor average service of 17ms per operation. Now that's going to slow things down. The cure here is quite simple - there's too much activity in /tmp - get the applicaiton guys to move it to a partition local to the application and the problem will be lessened.

avserv[ sd1,h ] = 0
avserv[ sd0 ] = 17
avserv[ sd320 ] = 2
avserv[ sd30 ] = 0
avserv[ sd1 ] = 17
avserv[ sd330 ] = 0

busy[ sd0 ] = 28
busy[ sd320 ] = 0
busy[ sd230 ] = 0
busy[ sd203 ] = 0
busy[ sd30 ] = 0
busy[ sd1 ] = 29

rwpersec[ sd0 ] = 57
rwpersec[ sd320 ] = 1
rwpersec[ sd230 ] = 0
rwpersec[ sd203 ] = 0
rwpersec[ sd30 ] = 0
rwpersec[ sd1 ] = 57

blkpersec[ sd0 ] = 920
blkpersec[ sd320 ] = 10
blkpersec[ sd230 ] = 0
blkpersec[ sd203 ] = 0
blkpersec[ sd30 ] = 0
blkpersec[ sd1 ] = 909

avwait[ sd0 ] = 7
avwait[ sd320 ] = 0
avwait[ sd230 ] = 0
avwait[ sd203 ] = 0
avwait[ sd30 ] = 0
avwait[ sd1 ] = 7

Ajowan ke paratha

Time for more geek food. Parathas are very much under appreciated in Asian cuisine that's presented to the restaurant going public - you'd be hard pressed to get a fresh paratha anywhere I've eaten.

This recipe comes from an book I picked up at a jumble sale:

Lot 1

8oz wholewheat flour
2 tsp Ajowan seeds
3/4 tsp salt
1/2 tsp chilli powder
7 fl oz water

Lot 2

3 tbsp melted ghee

Method

• Mix lot 1 in a bowl to a dough, kneading well. Leave for 15 mins for the dough to rest.
• Divide into 8 - 10 lumps

Loop

• On a floured surface, roll out each ball into an oblong, 3 inches by 6-8 inches
• Paint top surface with some of Lot 2
• Fold one third of the length into the middle, fold the other third onto this
• Repeat loop three times
• Roll out to desired size (3-4 x 6-8 inches ideally)
  

Cook

• Cook in a dry frying pan until surface bubbles and turns slightly opaque ( no ghee)
  
• Paint surface with some of Lot 2
• turn over and cook for a minute or so and repeat on other side

Eat with any curry or dall.