DeepPeep: discover the hidden web
DeepPeep is a search engine specialized in Web forms. The current beta version currently tracks 13,000 forms across 7 domains. DeepPeep helps you discover the entry points to content in Deep Web (aka Hidden Web) sites, including online databases and Web services.
Moteur de recherche pour le web invisibleNew Search Technologies Mine the Web More Deeply - NYTimes.com
"Now a new breed of technologies is taking shape that will extend the reach of search engines into the Web’s hidden corners. When that happens, it will do more than just improve the quality of search results — it may ultimately reshape the way many companies do business online."
Google now indexes a trillion web pages - but that's just a fraction of what's out there. So, what does it miss?
...google is built for a static web...................New Search Technologies Mine the Web More Deeply - NYTimes.com
An interesting look at the daunting task of connecting/mining the interwebs.
Search engines are starting to penetrate databases that are set up to respond to typed queries.
how to search databases, semantic sebThe dark side of the internet | Technology | The Guardian
26 Nov 09: new article which gives idea of the scale of the Deep Web and how much of it is actually searched...!
ut Rajaraman knows different. "I think it's a very small fraction of the deep web which search engines are bringing to the surface. I don't know, to be honest, what fraction. No one has a really good estimate of how big the deep web is. Five hundred times as big as the surface web is the only estimate I know."
om hippy bible the WhGoogle: "We're Not Doing a Good Job with Structured Data" - ReadWriteWeb
That's something that's a bit troublesome - if better search technology for indexing the Deep Web comes into existence outside of Google, the world may not end up using it until such point Google either duplicates or acquires the invention.
Enabling a Google-like search from structured sources (databases)
Google and Yahoo approaching structued Web
Halevy, who heads the "Deep Web" search initiative at Google, described the "Shallow Web" as containing about 5 million web pages while the "Deep Web" is estimated to be 500 times the size. This hidden web is currently being indexed in part by Google's automated systems that submit queries to various databases, retrieving the content found for indexing. In addition to that aspect of the Deep Web - dubbed "vertical searching" - Halevy also referenced two other types of Deep Web Search: semantic search and product search.10 Search Engines to Explore the Deep End of the Invisible Web
he Invisible Web refers to the part of the WWW that’s not indexed by the search engines. Most of us think that that search powerhouses like Google and Bing are like the Great Oracle…they see everything. Unfortunately, they can’t because they aren’t divine at all; they are just web spiders who index pages by following one hyperlink after the other. But there are some places where a spider cannot enter. Take library databases which need a password for access. Or even pages that belong to private networks of organizations. Dynamically generated web pages in response to a query are often left un-indexed by search engine spiders.