Wednesday, June 09, 2010

New Index Engine by Google

Content on the web grows faster than everyone can imagine. Even Google is having problem to index all this content using his old search index engine. That's why they built a new seach index engine and they called it Caffeine.

Today, they officially announced it on the Google Blog. The basic idea of Caffeine is like a real time updater for any content on any web. Any small changes on your web will be known by the indexer and it will add it to the index database on Google's server. Thus bringing the latest content to people's search in Google's website Rock

Google has also provide a small information about how it works. It is a cool system indeed.

Taken from Google Blog:
Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day. You would need 625,000 of the largest iPods to store that much information; if these were stacked end-to-end they would go for more than 40 miles.
You can see how the old and new search index works in the illustration above.

Here's a video about how a search process works in Google:

Update (10 seconds later): I have seen this information listed in Google's indexing. That was very fast. It really works Yahoo Banana Rock RockApplause