Search spiders have always stumbled
- Cory Doctorow
- Uncategorized
- Nov 26, 2001
Search spiders have always stumbled upon documents that Webmasters had thought secret, but now that Google’s adding the ability to index complex document formats like PDF, even more password files, love notes and dmaning memos are showing up online.
“Our specialty is discovering, crawling and indexing publicly available information,” said Google spokesman David Krane. “We define public as anything placed on the public Internet and not blocked to search engines in any way. The primary burden falls to the people who are incorrectly exposing this information. But at the same time, we’re certainly aware of the problem, and our development team is exploring different solutions behind the scenes.”