2. MAKING SENSE OUT OF
THINGS ON THE WEB
@pradeepbv

3. We have been accumulating
a lot of information
3

4. http://en.wikipedia.org/wiki/File:Jingangjing.jpg 4

5. http://en.wikipedia.org/wiki/File:Printer_in_1568-ce.png
http://en.wikipedia.org/wiki/File:BuxheimStChristopher.jpg 5

6. 6
http://en.wikipedia.org/wiki/Odhecaton

7. What hath God wrought
http://upload.wikimedia.org/wikipedia/commons/f/f1/The_First_Telegraph.jpg
7

8. 1891 Telegraph Lines
http://en.wikipedia.org/wiki/File:1891_Telegraph_Lines.jpg
8

9. Mr Watson—Come here
I want to see you
9
http://www.boerner.net/jboerner/?p=9396

10. radio
Radio
10

11. http://www.elon.edu/e-web/predictions/150/1930.xhtml 11

12. 12

13. 13

14. 14

15. www
15

16. http://en.wikipedia.org/wiki/File:NCSA_Mosaic.PNG
16

17. the Internet had an estimated
16 million users by 1995
17

18. http://en.wikipedia.org/wiki/Venture_capital 18

19. People from all over the world
started sharing their interests,
hopes and dreams online
19

20. 20

21. 21
http://electrokami.com/wp-content/uploads/2010/09/the-internet-in-real-life.jpg

22. The number of devices connected to IP networks
will be nearly three times as high as the global
population in 2016
22

23. kilo
mega
tera
giga
pita
The Zettabyte Era exa
zetta
9,444,732,965,739,290,427,392 bits (1024 exbibytes) yotta
http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/VNI_Hyperconnectivity_WP.html 23

24. “Reports that say that something hasn't
happened are always interesting to me, because
as we know, there are known knowns; there are
things we know we know. We also know there
are known unknowns; that is to say we know
there are some things we do not know. But
there are also unknown unknowns – the ones
we don't know we don't know.”
Donald Rumsfeld, US Defense Secretary
at a press conference at NATO
Headquarters, Brussels, Belgium, June 6, 2002
Image: planetization.org
24

25. Nicholas Carr worries
that the flood of digital
information is changing
not only our habits, but
even our mental
capacities: Forced to scan
and skim to keep up, we
are losing our abilities to
pay sustained
attention, reflect
deeply, or remember
what we’ve learned.
25

26. Information overload?
http://blogs.tusc.k12.al.us/bhslibrary/files/2012/01/Information_overload.jpg 26

27. DO YOU KNOW WHAT ARE YOU LOOKING FOR?
27
http://www.teachersdiary.com/.a/6a0115703931fc970c0128765537ba970c-800wi

28. DO YOU KNOW WHERE TO FIND WHAT YOU WANT?
http://www.flickr.com/photos/special/1597251/ 28

29. REGULAR SEARCH #FAIL?
http://www.flickr.com/photos/sumrow/1267682594/sizes/l/ 29

30. IS THERE A SUPERHERO
WHO CAN HELP?
http://www.flickr.com/photos/sumrow/1267682594/sizes/l/ 30

31. BUILD YOUR OWN
SEARCH SERVICE
Yes, you are the superhero

32. BOSS IS
BUILD YOUR OWN
SEARCH SERVICE
http://developer.yahoo.com/search/boss/

33. BOSS PROVIDES APIS
TO OUR SEARCH
DATA STORES

34. TO BUILD YOUR OWN
POWERFUL
SEARCH APPLICATIONS

35. BOSS allows you to
search over
Web, images,
news & Blogs

36. You can even monetize your
applications using Search Ads
from BOSS and get support.

37. What can be done on top of BOSS?
• Blend and re-rank search results
• Your own look and feel
• Mix it with other APIs

38. BOSS Pricing

39. Free for building your hacks!!

40. Where do I start?

41. What’s in it?
Restful XML and JSON API
Web
Image
Spelling
News
Search Ads
http//www.flickr.com/photos/joeshlabotnik/419914250/sizes/o/in/photostream/.jpg

42. Oauth based Autentication
http//www.flickr.com/photos/friarsbalsam/5736126308/sizes/o/in/photostream/.jpg

43. What else do I get?
 Web and Limited Web results
 Image attributes
like height, width, etc
 Time span filtering
for News Search
 Document type filtering
 Extended abstracts
http//www.flickr.com/photos/acidpix/6021203584/sizes/o/in/photostream/.jpg

44. BOSS + YQL
• Table Name: boss.search Example
Parameters
Consumer Key ck -
Consumer Secret secret -
Query Term q ‘iitd’
• e.g. select * from boss.search where ck=… and
secret=… and q=‘openhackindia’

45. Searching “The Dark Knight”

47. Finding images of “The Dark Knight Rises”
select * from boss.search where q="The Dark
Knight Rises" and service="images" and
ck="..." and secret="..."

48. Finding “The Dark Knight Rises” in IMDB,
movies.yahoo.com
select * from boss.search where q="The Dark
Knight Rises" and
sites="imdb.com,movies.yahoo.com" and
ck="..." and secret="..."

49. Spell Check and Correction
select * from boss.search where q="The Dark
Knight Rises" and service="spelling" and
ck="..." and secret="..."

50. Finding news on “The Dark Knight Rises”
select * from boss.search where q="The Dark
Knight Rises" and service="news" and ck="..."
and secret="..."

51. And through the BOSS API
 Getting multiple data sets
 /ysearch/web,images,news?q=anna
 /ysearch/web,images,news?web.q=anna&images.q=anna&news.q=lokpal
 Searching through sites
 A Simple Movie Search
 /ysearch/web?q=“Dark Knight”&
sites=movies.yahoo.com,netflix.com,imdb.com
 AND/OR operators
 /ysearch/web?q="steve jobs"AND((ipad)OR(iphone))&sites=bestbuy.com,newegg.com
 Important: Use Braces or quotes

52. Unary Operators
 Search for Batman but not “Dark Knight”
 q=(batman -“Dark Knight")
 Find pages with “Heath Ledger” but not “Dark
Knight”
 q=+”heath ledger”–”Dark
Knight”&sites=movies.yahoo.com
 Force auto-spelling off
 q=+”drk knight”

53. Searching in body and in title
 Searching for Dark Knight in the Title on Yahoo movies
 q=reviews intitle:"dark knight"&sites=movies.yahoo.com
 Searching for Dark Knight in the Title in Yahoo movies
containing Christian Bale
 q=reviews intitle:"dark knight" inbody:"christian
bale"&sites=movies.yahoo.com

54. Market and document specific Filters
 Search for “Dark Knight” in India specific sites
 q=“Dark Knight”&market=en-in
 Search for “PDF’s containing “Dark Knight”
 q=“Dark Knight”&type=pdf
 Search for MS Office type (except PPT’s)
containing “Dark Knight”
 q=“Dark Knight”&type=msoffice,-ppt

55. Output

56. Image search parameters
 Search for images that are not offensive
 /ysearch/images?q=“san francisco”&filter=yes
 Search for images that are wallpaper size
 /ysearch/images?q=“san francisco”&dimensions=wallpaper
 Search for a image at a certain refer URL
 /ysearch/images?q=yahoo&refererurl=http://www.flickr.com
• Interesting Output Fields
 format, file size, height, width, title, total result count

57. News search parameters
 Search news that is less than 7 days old
/ysearch/news?q=lokpal&age=7d
Search news that is between 20hrs and 2 days old
/ysearch/news?q=lokpal&age=20h2d
Re-rank news results by date
/ysearch/news?q=lokpal&ranking=true
Interesting Output Fields
 Source, Date, Source URL

58. EXAMPLE HACKS

59. Duckduckgo.com

60. Interceder

61. Ask-boss (v1)
Hack: http://ask-boss.appspot.com
Code: https://github.com/saurabhsahni/Hacks/tree/master/askBOSS

62. webmeme.in

63. http://hackyourworld.org/~iitb_pacman/search/

64. I did BOSS and got data, now how
to extract information of out it?

65. make sense out of it?

67. Content Analysis
select * from contentanalysis.analyze where
text="Yahoo! kicks off hackday”

68. Content Analysis from a URL
select * from contentanalysis.analyze where
url="http://www.cnn.com/"

69. Term Exraction
select * from search.termextract where context
in (select description from rss where url=‘’)

70. More resources
 Yahoo! BOSS: http://developer.yahoo.com/boss
 BOSS Technical Documentation:
http://developer.yahoo.com/search/boss/boss_api_guide/
 YQL: http://developer.yahoo.com/yql
 Amazon Web Services: http://aws.amazon.com
 oAuth: http://oauth.net/
 Open Data: http://theinfo.org
 Alt Search Engines: http://www.altsearchengines.com/

71. Happy hacking!