How to prevent robots (Google Bot, MSNBot, Yahoo Slurp! etc.) from displaying cached pages of your site

August 23rd, 2006 Search engines like Google often fetch your site description from ODP (Open Directory Project) description instead of fetching it from the site content or meta-tags. This may not always be desirable as it may not represent your site properly.

July 7th, 2008 Google search bot (Googlebot) parses robots.txt file to find excluded sections of the site like any good webbots. However unlike the other bots, Google bot behaves differently when you have a section for all bots (*) as well as a section specifically for Googlebot.

February 27th, 2006 Google has newly introduced Google Pages which allows you to create your webpages with AJAX interface (Web 2.0?). It is quite a slick product, provides 100MB of space, file uploading etc.

June 27th, 2007 Translator Plugin Pro & Gold tries through all the available engines for a language combination to translate the page. This ensures much higher reliability than single engine translations.

July 25th, 2008 Much has been written about Yahoo Pipes and lots of how-to tutorial. I was pleasantly surprised to see the ease of use of Yahoo Pipes and how easily I could create a simple pipe to get data in RSS feed format.

November 4th, 2006 This is old news for most of us and yet when I wanted to quickly find it, it took some searching. So here it is, in simple language, what you can do to prevent your web pages from being cached by browsers.

February 23rd, 2008 I recently came across an instance of Google Proxy hacking with one of my clients, which removed his index page and other pages from SERP (Search Engine Ranking & Positioning) and he lost the page ranks (went down to zero). We were asked to protect his site against Google Proxy hacking, a really dangerous technique which can not only cause you to loose page rank but also remove your pages from SERP, all because Google cannot properly identify original pages from duplicates.

July 24th, 2006 Many online blogging software and other products require some type of cron and / or asynchronous execution functionality. Spammers and web robots are an essential part of any website today.

February 24th, 2006 Google seems to have developed a habit of introducing new services and then withdrawing them due to high demand and reintroducing them selectively later. Google Web Accelerator, Google Analytics and now Google Pages, all follow this overused pattern.

November 12th, 2007 AdBlock Plus is a controversial Firefox plugin to prevent ads from displaying on web pages. There is a simple procedure to detect (and take appropriate action) AdBlock Plus.

May 24th, 2006 Rumors are rife about Google's venture into lucrative online music store market, thereby competing with Apple's highly successful iPod. Declan finds that Google has added /music to its robots.txt file, potentially indicating the site can come up anytime.

June 9th, 2008 We have developed a new combo bar for displaying the languages for Translator Plugin Pro & Gold. Check it on the top right of this blog.

September 24th, 2007 A simple google search reveals over 55.5 million pages are generated using WordPress today. The WordPress search volume shows a healthy increase over time as can be seen from the graph.

July 1st, 2007 I use both YPN & AdSense. Read below for a summary of my experience with both Google AdSense and Yahoo Publisher Network (YPN).

July 28th, 2008 Yahoo Pipes and Google Mashup Editor both allows you to create mashups with RSS feeds. The following is a brief comparative review.