Friday, May 11, 2007

Grid Computing Slowly Falls From Grace

I spoke at the The 451 Groups ECS Summit, London Regional Roundtable on Thursday, May 10, 2007 at the Brewery at Chiswell Street in London where the focus was supposed to be on Grid computing, the focus of much research by the 451 Group over the last year.

I noticed the trend in early 2006 that term Grid was losing it market appeal. It started disappearing from job titles, to be replaced by High Performance Computing (HPC) - this has now become mainstream with at least two IB's having the post. Another two trends were noted - data architecture is now high on the agenda and low-latency messaging is seen as the crucial to meeting the challenges of high transaction volumes.

So where's Grid computing heading? The general concensus were that the technology had not lived up to its promises in terms of performance and in particular: managability, security and lack of data architecture seemed to be the prime areas of concern. Grid in IB needs to grow up - in essence, despite the marketing hype, current grid offerings are little more than cluster computing applied to a few niche areas in IB such as CDO/CDO squared (btw check out www.cdo2.com - run by an acquaintance of mine - a real web service with real customers running on BLAST

The reality is that most jobs can be done now on multi-core, large memory machines. An 8-core/4 CPU machine with 64GB memory are a lot cheaper than an investment in grid - and you get a lot of processing power now for not a lot of cash. Nodes can be chained together with 1GE nics and switch to produce an effective HPC cluster and you can either use your own threaded app with memcached or have a play with jini.



1 comment:

Andy Hospodor, Ph.D. said...

Good observation - Grid Computing is difficult to say in a sentence. The most common response "huh?"

High Performance computing works because people get the value of High Performance and know what computing means.

Grid computing and High Performance computing will converge in time. Only the government can afford to build megasize homogenous datacenters. For now, it is easier for industry to deploy dedicated SMP systems that are consistent with the trend in server consolidation. But soon, they will realized the opportunity and harvest CPU cycles. Companies like JP Morgan and others in financial services (especially risk analysis) lead this charge.

As for the rest of the world, well, many are left wondering about web services and when they will replace EDI flat files in markets such as publishing and textbook distribution.