Google only searches 1% of the data, use the invisible web to search the other 99% of data

1. Michelle Post, Ph.D.

2.  What is it?
 Visible vs. Invisible
 How does it work?
 How big is it?
 Where is it going?
 Invisible Web Search Engines
 Invisible Web Databases
 Video Training
 Resources
 References
© Michelle Post, Ph.D. 2

3.  “Searching on the Internet today can be compared to
dragging a net across the surface of the ocean. While
a great deal may be caught in the net, there is still a
wealth of information that is deep, and therefore,
missed. The reason is simple: Most of the Web's
information is buried far down on dynamically
generated sites, and standard search engines never
find it” (Bergman, 2001, para. 2).
 “The "visible web" is what you can find using general
web search engines. It's also what you see in almost
all subject directories. The "invisible web" is what you
cannot find using these types of tools” (UC Berkeley
Library, n.d., para. 1).
© Michelle Post, Ph.D. 3

4.  “The deep Web contains 99% of the
information content of the Web; however,
most of this information is contained in
databases and is not indexed by search
engines” (Gruchawaka, 2010, para. 1).
 “If the most coveted commodity of the
Information Age is indeed information, then
the value of deep Web content is
immeasurable” (Bergman, 2001, para. 5).
Michelle Post, Ph.D. 4

5. Surface Web Deep Web
Millions of web pages Over 200,000 databases
1 billion documents 550 billion documents
19 terabytes 7,750 terabytes
Broad shallow coverage Deep vertical coverage
Results contain ads Results contain no ads
Content unevaluated Content evaluated by experts
(Bergman, 2001)
© Michelle Post, Ph.D. 5

6.  “The deep Web is qualitatively different from
the surface Web. Deep Web sources store
their content in searchable databases that
only produce results dynamically in response
to a direct request” (Bergman, 2001, para. 4).
© Michelle Post, Ph.D. 6

7.  Public information on the Deep Web is currently 400 to
550 times larger than the commonly defined World Wide
Web.
 The Deep Web contains 7,500 terabytes of information,
compared to 19 terabytes of information in the surface
Web.
 The Deep Web contains nearly 550 billion individual
documents compared to the 1 billion of the surface Web.
 More than an estimated 200,000 Deep Web sites presently
exist.
 Sixty of the largest Deep Web sites collectively contain
about 750 terabytes of information – sufficient by
themselves to exceed the size of the surface Web by 40
times. (CompletePlanet, n.d., para. 6)
Michelle Post, Ph.D. 7

8.  “The deep Web is the fastest growing sector
of the Web and it appears to be the „paradigm
for the next generation Internet‟”
(BrightPlanet, 2005, para. 35).
© Michelle Post, Ph.D. 8

9.  ResourceShelf -
http://www.resourceshelf.com/
 Direct Search -
http://www.freepint.com/gary/direct.htm
 Complete Planet -
http://www.completeplanet.com/
Michelle Post, Ph.D. 9

10.  Listing of Largest Deep Web Sites -
http://aip.completeplanet.com/aip-
engines/help/largest_engines.jsp
 Directory of Open Access Journals -
http://www.doaj.org/
 Educator's Reference Desk - http://www.eduref.org/
 Find Articles - http://www.findarticles.com/
 On-Line Encyclopedia of Integer Sequences -
http://www.research.att.com/~njas/sequences/
 PubMed -
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi
 Super Searchers Web Page -
http://www.infotoday.com/supersearchers/
Michelle Post, Ph.D. 10

11.  The Virtual Private Library and Deep Web -
http://www.dailymotion.com/video/x9udyo_t
he-virtual-private-library-and-dee_news
Michelle Post, Ph.D. 11

12. 100 Time-Saving Search Engines for Serious Scholars -
http://www.onlineuniversities.com/blog/2010/03/100-
time-saving-search-engines-for-serious-scholars/
Deep Web FAQ - http://aip.completeplanet.com/aip-
engines/help/help_deepwebfaqs.jsp
The Invisible Web Databases -
http://www.noodletools.com/debbie/literacies/informatio
n/5locate/advicedepth.html
Those Dark Hiding Places: The Invisible Web Revealed -
http://library.rider.edu/scholarly/rlackie/Invisible/Inv_We
b.html
White Paper: The Deep Web: Surfacing Hidden Value -
http://quod.lib.umich.edu/cgi/t/text/text-
idx?c=jep;view=text;rgn=main;idno=3336451.0007.104
Michelle Post, Ph.D. 12

13. Bergman, M. (2001, August). White Paper: The deep web: Surfacing
hidden value. The Journal of Electronic Publishing, 7(1), doi:
10.3998/3336451.0007.104. Retrieved from
http://quod.lib.umich.edu/cgi/t/text/text-
idx?c=jep;view=text;rgn=main;idno=3336451.0007.104
BrightPlanet. (2005). Deep webfaq. Retrieved from
http://www.brightplanet.com/
CompletePlanet. (n.d.). How large is the deep web. Retrieved from
http://aip.completeplanet.com/aip-
engines/help/help_deepwebfaqs.jsp
Gruchawka, S. (2010). Using the deep web. Retrieved from
http://techdeepweb.com/
UC Berkeley. (n.d.). Invisible or deep web: What it is, how to find it,
and Its inherent ambiguity. Retrieved from
http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Invisi
bleWeb.html
© Michelle Post, Ph.D. 13