Khipu-pooing Mainstream Media

It would hardly be fair and balanced for me to keep touting the virtues of reading Wired without also pointing out occasional errors as I see them:

In "Untangling the Mystery of the Inca" they say...

Type 'baseball' into Google and its spiders will race over the Internet, look at links, and spit back that is the 11th most useful site for you and is the 22nd.


This makes it sound like search engines are on-demand botnets or something, and it's wrong. Search engines are constantly updating their caches of sites, i.e. taking an ongoing snapshot of the Web. Their crawlers are a critical part of this, and they can be attracted to sites by actions taken by Webmasters in some ways, but they work by their own clocks as far as users are concerned. Their behaviors are proactive, and they check on sites at depths and intervals in accordance with sites' content change frequency, inbound link popularity and other factors. Users are served information based on what search engine indexes (made of caches) determine to be relevant returns, but engine spiders are not reactive dogs who fetch the live Web whenever users query their masters for information (bonus points to anyone who remembers Lycos perpetuating this by implication back in the day).

This is why we have tactical options like cloaking, which is about getting something into the search engines' caches that differs from what human visitors are shown.

Spread His Word

    About this entry