Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (13)
Semelhante a [@IndeedEng] Large scale interactive analytics with Imhotep
Semelhante a [@IndeedEng] Large scale interactive analytics with Imhotep (20)
Mais de indeedeng
Mais de indeedeng (8)
Último
Último (20)
[@IndeedEng] Large scale interactive analytics with Imhotep
- 2. Large Scale Interactive Analytics with Imhotep
- 7. What is Imhotep? Imhotep is a highly scalable analytics architecture for querying faceted datasets
- 8. Open sourcing Imhotep Imhotep will be an OPEN SOURCE highly scalable analytics architecture for querying faceted datasets
- 13. A Brief History of Analytics @Indeed
- 14. What's best for the job seeker?
- 17. Query
- 18. Query Location
- 20. Title: Front End Software Engineer Position: 1 Clicked: 0 Country: US Query: indeed software engineer Location: austin Timestamp:2014-04-30T20:00:00 Organic Impression Log Entry
- 21. Analytics on Raw Logs
- 22. Ramses
- 23. ● Search logs ● Extract metrics from matches ● Graph aggregated metrics Ramses
- 24. ● Search logs ● Extract metrics from matches ● Graph aggregated metrics Input -> Query and Metric Output -> Aggregated metrics by bucket Ramses
- 25. How many organic clicks did we have in Australia?
- 26. QUERY country:au METRIC organic_clicks How many organic clicks did we have in Australia?
- 27. How many organic clicks did we have in Australia?
- 28. Does test group A or B have more revenue?
- 29. QUERY testgroup:A, testgroup:B METRIC revenue Does test group A or B have more revenue?
- 30. Does test group A or B have more revenue?
- 31. How has traffic from Yahoo! changed over time in Great Britain, Germany, and Japan?
- 32. QUERY from:yahoo AND country:(gb, de, jp) METRIC visits How has traffic from Yahoo! changed over time in Great Britain, Germany, and Japan?
- 33. How has traffic from Yahoo! changed over time in Great Britain, Germany, and Japan?
- 34. ● How many unique queries in the US? ● What are the top 50 queries in the US? ● How many clicks did each of those queries receive? Questions Ramses can’t answer
- 35. Imhotep
- 36. Began as a distributed iteration and group-by engine for building click prediction models. Imhotep Origins
- 37. We use an iterative algorithm to build decision trees level-by-level. Decision Tree Builder
- 39. Began as a distributed iteration and group-by engine for building click prediction models. Leveraged ability to do massive group-bys and aggregates to make real-time analytics engine. Imhotep Origins
- 40. How many Android App users with accounts older than 30 days saved at least 1 job in the past week?
- 41. What titles have the highest click-through rate for the query “Architecture” in the US? What about the lowest click-through rate?
- 42. For job seekers who click on Google jobs in Ireland, what other company’s jobs do they click on?
- 45. Marketing Sciences Research, analysis, and automation team supporting marketing initiatives
- 46. Imhotep Imhotep is a highly scalable, [soon to be] open source, analytics architecture for querying faceted datasets
- 48. Imhotep@Indeed Ad hoc exploration Specific analysis
- 49. Imhotep@Indeed Ad hoc exploration Specific analysis Extensible infrastructure
- 50. Ad hoc exploration Public Crunchbase Dataset Source: CrunchBase CrunchBase 2013 Snapshot © 2013
- 51. Ad hoc exploration Public Crunchbase Dataset Document Source: CrunchBase CrunchBase 2013 Snapshot © 2013
- 52. Ad hoc exploration Public Crunchbase Dataset Fields Source: CrunchBase CrunchBase 2013 Snapshot © 2013
- 53. Ad hoc exploration Public Crunchbase Dataset Metric Source: CrunchBase CrunchBase 2013 Snapshot © 2013
- 54. Interactive tool for exploring Imhotep data Imhotep Data Explorer
- 55. Interactive tool for exploring Imhotep data Also: a badass hyperlinked pivot table Imhotep Data Explorer
- 56. Imhotep is Large Scale Total size of all indexes: 125TB Jobsearch index (largest): 30TB ● Over 48 billion documents
- 58. Query
- 59. Query Location
- 61. Organic Impression A job that was displayed as the result of a search
- 63. Title
- 65. Description
- 66. Job Age
- 68. Organic Impression Document Title: Front End Software Engineer Position: 1 Clicked: 0 Country: US Query: indeed software engineer Location: austin Timestamp:2014-04-30T20:00:00
- 69. Organic Impression Index Title: Front End Software Engineer Position: 1 Clicked: 0 Country: US Query: indeed software engineer Location: austin Timestamp:2014-04-30T20:00:00
- 70. Imhotep Data Explorer can’t... Combine results from multiple datasets
- 71. Combine results from multiple datasets Be easily automated Imhotep Data Explorer can’t...
- 73. IQL - Imhotep Query Language Can combine results from multiple datasets Allows for automation of data tools
- 74. IQL queries - requirements Index Date range Metrics
- 75. IQL queries - optional Index Date range Metrics Filters Group by
- 76. IQL - Metrics select count() from organic ‘2013-12-05’ ‘2013-12-10’ where country=ie and clicked=1 group by companyid Metrics
- 77. select count() from organic ‘2013-12-05’ ‘2013-12-10’ where country=ie and clicked=1 group by companyid IQL - Indexes Index
- 78. select count() from organic ‘2013-12-05’ ‘2013-12-10’ where country=ie and clicked=1 group by companyid IQL - Date Range Date Range
- 79. select count() from organic ‘2013-12-05’ ‘2013-12-10’ where country=ie and clicked=1 group by companyid IQL - Filters Filters
- 80. select count() from organic ‘2013-12-05’ ‘2013-12-10’ where country=ie and clicked=1 group by companyid IQL - Filters Groups
- 81. IQL Question Do companies that have raised more than $10 million in the Austin get more clicks on average than those raised less than $10 million?
- 82. Methodology 1) organic index: select companies in the US which received organic clicks
- 83. Methodology 1) organic index: select companies in the US which received organic clicks 2) crunchbase index: select companies, and the amount of funding for companies receiving investments in Austin
- 84. Methodology 1) organic index: select companies in the US which received organic clicks 2) crunchbase index: select companies, and the amount of funding for companies receiving investments in Austin 3) Join, segment, and do the math!
- 87. Large Scale Interactive Analytics Platform ● 123 Unique Indexes ● Largest Index 30TB ● Total size ~125TB
- 88. Large Scale Interactive Analytics Platform IQL -> Largely Programmatic access ● approx 76k queries/day ● Avg time to execute 0.67 seconds Ramses -> Largely Human ● approx 3,400 queries/day ● Avg time to execute 4.4 seconds
- 89. Large Scale Interactive Analytics Platform Users ● 198 unique users in past month ● 25,622 unique queries in past month ● Avg 53 queries/user per day
- 90. Large Scale Interactive Analytics Platform 40+ internal clients ● 6 Analytics Webapps ● 5 dashboards ● 10 programming/scripting shells ● 6 monitoring apps ● … and more
- 91. Large Scale Interactive Analytics Platform One Tool-set for all data ● Website usage ● Operational Monitoring ● Financial Reporting ● Google Analytics ● Internal Webapp Usage ● External Reports
- 92. Solving a real problem
- 93. Providing the Best Results Show the jobs that users are most interesting to our users
- 94. Providing the Best Results Clicks are a very good indicator of interest
- 95. Providing the Best Results Clicks are a very good indicator of interest More clicks -> More Relevant Less clicks -> Less Relevant
- 96. Architecture Very hard query to serve correctly
- 97. Architecture Very hard query to serve correctly Architecture terminology has been co- opted by technology
- 98. Terminology Common to both Software and Architecture Blueprint Design Framework Infrastructure Engineer Project manager Development Technical architect Software Modeling Computation Code reviews
- 99. Architecture vs Software Titles Architect CAD Designer Project Manager vs Software Architect UI Designer Project Manager
- 100. Query Management Indeed uses Imhotep to improve matching
- 101. Query Management Indeed uses Imhotep to improve matching Automatically detect results that should be added or removed from queries
- 102. Query Management Indeed uses Imhotep to improve matching Automatically detect results that should be added or removed from queries 26,790 rules across all countries
- 103. Imhotep Open Source Imhotep Open Source ETA: August 1, 2014
- 104. Imhotep Open Source Follow along at our blog engineering.indeed.com Sign up for mailing list to get latest updates go.indeed.com/imhotep-announce
- 105. Q & A
- 106. Next @IndeedEng Talk Launching Indeed Around the World Davide Novelli, International Director David Tulig, Tech Lead May 28, 2014 http://engineering.indeed.com/talks
- 107. More Questions? Jason David James Jeff