Personalizing Search at LinkedIn
•Transferir como PPTX, PDF•
2 gostaram•1,646 visualizações
Bay Area search meetup (June 2015)
Recomendados
Recomendados
Mais conteúdo relacionado
Destaque
Semelhante a Personalizing Search at LinkedIn
Semelhante a Personalizing Search at LinkedIn (20)
Último
Último (20)
Personalizing Search at LinkedIn
- 1. Recruiting SolutionsRecruiting SolutionsRecruiting Solutions Ganesh Venkataraman Viet Ha-Thuc Personalizing Search @ LinkedIn
- 2. Bigger Picture ▪ LinkedIn’s vision – Create economic opportunity for every member of the global workforce ▪ Connect members to other members, knowledge and opportunity and help them be great at what they do
- 3. Economic Graph ▪ Organize people, companies, jobs, knowledge and map out the economic graph 3
- 4. Role of Search ▪ At the heart of the economic graph, search makes the economic graph accessible, useful and actionable ▪ Powers searching people, jobs, companies, schools etc. ▪ On linkedin.com consumer, recruiter, sales solutions 4
- 6. Basic Nomenclature 6 TypeAhead/TYAH Full Search/SERP
- 8. 8
- 9. Search is about understanding the user intent 9
- 10. LinkedIn Search - An Overview 10 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 11. Let’s talk intent - Navigational ▪ Navigational - exactly one result in mind 11
- 12. Two types of Intent - Exploratory ▪ Exploratory - Typically more than one entity in mind 12
- 13. How to handle navigational queries? Be Fast Type Less Be Lenient 13
- 14. Handling Navigational Queries ▪ Type Less – Index prefixes (‘ga’, ‘gan’, ‘gane’ => ‘ganesh’) ▪ Be Fast – Do not retrieve all documents – Order documents in posting list by static rank – Modify query for targeted retrieval ▪ Be Lenient – Smart spell correction 14
- 15. Exploratory Queries ▪ If possible guide users to more structured queries ▪ Above query could go into different verticals if these are selected ▪ User intent becomes much clearer 15
- 17. Unclear intent - Federating TYAH results 17
- 18. LinkedIn Search - Bird’s eye view 18 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 19. Query Processing - things not strings 1919 TITLE CO GEO TITLE-237 software engineer software developer programmer … CO-1441 Google Inc. Industry: Internet GEO-7583 Country: US Lat: 42.3482 N Long: 75.1890 W (RECOGNIZED TAGS: NAME, TITLE, COMPANY, SCHOOL, GEO, SKILL )
- 20. Retrieval ▪ Custom search engine to handle 100’s of millions of documents (Galene) ▪ Key Features: – Offline indexing pipeline – Supports live updates with fine granularity – Static Ranking ▪ Posting list organized by static rank for each document ▪ Enables early termination 20
- 21. LinkedIn Search - Bird’s eye view 21 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 22. Ranking ▪ Manually tuning vs. Learning to Rank (LTR) ▪ Why Learning to Rank? – Hard to manually tune with very large number of features – Challenging to personalize – LTR allows leveraging large volume of click data in an automated way 22
- 23. Training Data: Human Label
- 24. What if the searcher is a job seeker? Or a recruiter? Training Data: Human Label
- 25. ▪ Relevance depends on who’s searching ▪ Difficult to scale Training Data: Human Label
- 26. Training Data: Click Stream Approach: Clicked = Relevant, Skipped = Not Relevant User eye scan direction Unfair penalized
- 27. Training Data: Click Stream Approach: Graded relevance Uncertain (middle level) Non-relevant Relevant
- 28. Feature Overview ▪ Textual features ▪ Social features ▪ Homophily features – Geo – Industry ▪ Inferred Searcher Interests ▪ etc.
- 29. Inferred Searcher Interests Interests * Locations * Industry ...
- 30. Learning Algorithm ▪ Coordinate Ascent Algorithm – Listwise approach ▪ Objective function: Normalized Discounted Cumulative Gain (NDCG) – Defined on graded relevance – Intuition: more useful to show more-relevant documents at higher positions
- 31. LinkedIn Search - Bird’s eye view 31 Query Processing Retrieval Ranking Federated Page Construction Search Assist ● Instant Results ● Guided suggestions ● Autocomplete suggestions Entity View/Action
- 33. ▪ Why do we need this? – Not to overwhelm the user with too much information –Make results personally relevant 33 Motivation
- 34. ▪ Challenges –Query can be ambiguous –Incomparability across vertical objects ▪Compare objects of different nature: individual job vs. people cluster ▪Objects associate with different signals 34 Motivation
- 36. Learning Federation Model ▪ Predicts: p(click| individual result OR vertical cluster, query, searcher) ▪ Training data: click logs ▪ Features –Relevance scores from base rankers –Searcher intent –Query intent –etc.
- 37. Features ▪ Searcher Intents – Mine searcher profiles and past behavior to infer intent ▪ Title recruiter -> recruiting intent ▪ Search for jobs -> job seeking intent – Machine-learned models predict member intents: ▪Job seeking ▪Recruiting ▪Content consuming 37
- 38. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions 38
- 39. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) 39
- 40. Features ▪ Query Intents: e.g. p(job vertical| “software engineer”) –Mine from historical searches and actions ▪ Personalized Query Intents –p(job vertical| “software engineer”, searcher) –Individual searcher → searcher group ▪p(job vertical| “software engineer”, job seeking searcher) 40
- 41. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable 41
- 42. Calibrate Signals across Verticals ▪ Relevance scores from vertical rankers are incomparable ▪ Construct composite features People relevance score of searcher if result is People f 1= ⎨0, otherwise 42
- 43. Calibrate Signals across Verticals ▪ Verticals associate with different signals 43 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 44. Calibrate Signals across Verticals ▪ Verticals associate with different signals 44 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 45. Calibrate Signals across Verticals ▪ Verticals associate with different signals 45 People Result Job Result Group Result Recruiting Intent Job Seeking Intent Content Consuming Intent
- 46. Conclusions ▪ Search personalization is at the core of our economic graph vision –Connect talent with opportunity at massive scale ▪ Click data is useful sources for personalized training data –Need to correct position bias ▪ Personalized features are keys ▪ Create composite features to calibrate across verticals