ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as naming, configuration management, synchronization, and group services - in a simple interface so you don't have to write them from scratch. You can use it off-the-shelf to implement consensus, group management, leader election, and presence protocols. And you can build on it for your own, specific needs.

Full article (876 words) »

Any reasonable (web) application which deals with some subset of web data (rss feeds, web pages, product pricing data etc.), has to use distributed data processing to go anywhere. Unless you have very deep pockets and / or have strong VC funding (which is rarer than a bottle of Mouton Rothschild Pauillac Premier Cru First Growth these days) you will have to opt for consumer grade hardware (instead of big iron which more companies opted during the dotcom era ) and use them in a distributed computing framework like hadoop or GridGain or older (more mature) high throughput computing systems like OpenPBS (ignore the marketing talk to download the free version) or Condor.

Full article (703 words) »