Daniel Tunkelang discusses three key areas - relevance, recommendations, and reputation - that LinkedIn is focusing on to improve the user experience. Relevance refers to improving search and discovery of profiles by combining different types of signals. Recommendations match members to jobs, groups, and other connections. Reputation involves identifying member expertise in skills and who the experts are in particular skills. Open problems remain around exploratory search, balancing exploration and exploitation of the data, and incentivizing the development of online reputation.

1. Daniel Keeping It Professional:Relevance, Recommendations, and Reputation. Daniel Tunkelang Principal Data Scientist at LinkedIn 1

3. Hard problems we’re tackling in:

4. Relevance

5. Recommendations

6. Reputation

7. Open problems2

8. Identity Connect, find and be found Rolodex, Resume, Business Card LinkedIn Profile, Address Book, Search Insights Be great at what you do Newspapers, Trade Magazines, Events Homepage, LinkedIn Today, Groups Everywhere Desktop Work wherever our members work Mobile, APIs, Plug-Ins What is LinkedIn? 3

9. Identity: Profile of Record 4

10. Identity: Connect with Others 5

11. Identity: Join the Conversation 6

12. Insights: Power of Aggregation Before employees worked at Yahoo! (169) Google (96) Oracle (78) Microsoft (72) IBM (43) Before employees worked at Google(475) Microsoft (448) LinkedIn (169) Apple, Inc. (154) ebay(133) 7

13. Insights: Market Research 8

14. Insights: Data Stories 9

15. Everywhere 10

17. People Search

18. Recommendations

19. Job Matching

20. Reputation

21. Skills11

22. People Search 12

24. 2B searches in 2010

25. Based on (cf. http://sna-projects.com/)13

26. People Search: Faceted Search 14

27. People Search: Network Facet 15

28. People Search: Type-Ahead 16

30. Network Rank, Profile Quality

31. Query-Dependent Signals

32. Field-Based Relevance

33. Personalized Signals

34. Network Distance17

35. People Search: Query-Independent Signals 18

36. People Search: Network Rank 19

37. People Search: Profile Quality 20

38. People Search: SEO 21

39. People Search: Query-Dependent Signals 22

40. People Search: Inferring Structure 23

41. People Search: Ambiguity vs. vs. 24

42. People Search: HMM + Segmentation for i in [1..n] s  w1 w2 … wi if Pc(s) > 0 a  new Segment() a.segs {s} a.prob Pc(s) B[i]  {a} for j in [1..i-1] for b in B[j] s wjwj+1 … wi if Pc(s) > 0 a  new Segment() a.segsb.segs U {s} a.probb.prob * Pc(s) B[i]  B[i] U {a} sort B[i] by prob truncate B[i] to size k 25

43. People Search: Personalized Signals 26

45. SIGIR 2011 Workshop on Entity-Oriented Searchhttp://research.microsoft.com/en-us/um/beijing/events/eos2011/

46. HCIR 2011 paper by Jonathan Koren on “Faceted Search Query Log Analysis” (forthcoming)http://hcir.info/hcir-2011/27

47. Job Matching 28

49. Job Description, Location, Similar Jobs, …

50. Candidate Features

51. Profile Data, Network, Activity, …

52. Standardization

53. Companies, Job Titles, Education, …29

54. … industry description functional area Current Position title summary tenure length industry functional area … Job Matching: Algorithm Corpus Stats Job Matching Transition probabilities Connectivity yrs of experience to reach title education needed for this title … Binary Exact matches: geo, industry, … Soft transition probabilities, similarity, … Text title geo company User Base Similarity (candidate expertise, job description) 0.56 Filtered Similarity (candidate specialties, job description) 0.2 Candidate Transition probability (candidate industry, job industry) 0.43 General expertise specialties education headline geo experience Title Similarity 0.8 Similarity (headline, title) 0.7 derived . . . 30

56. Complicates evaluation, training.

57. Important not to offend users.

58. e.g., by offering Peter Norvig a postdoc.

59. You can’t always get what you want

60. Every employer wants the hottest candidates.31

62. SIGIR 2011 paper by Cetintas et al. on “Identifying Similar People in Professional Social Networks”http://dl.acm.org/citation.cfm?id=2010123

63. Blog post on LinkedIn’s recommendation enginehttp://blog.linkedin.com/2011/03/02/linkedin-products-you-may-like/32

64. Skills 33

65. Skills: What are Skills? 34

66. Skills: Identifying Skills 35

67. Skills: Cluster and Disambiguate angel 36

68. Skills: Assigning Skills to People 37

69. Skills: Who are the Experts? 38

71. Combine query-independent, query-dependent, and personalized features.

72. Recommendations

73. Match people to jobs, groups, news, …

74. Reputation

75. Expertise relative to professional skills.39

76. ¿ Open Problems ? 40

77. Exploratory Search Exploratory Search Investigate Lookup Learn Accretion Analysis Exclusion/Negation Synthesis Evaluation Discovery Planning/Forecasting Transformation Fact retrieval Known item search Navigation Transition Verification Question answering Knowledge acquisition Comprehension/Interpretation Comparison Aggregation/Integration Socialize 41

78. Explore / Exploit 42

79. Incentives for Online Reputation 43

80. Thank You! Questions? Contact: dtunkelang@linkedin.com We’re Hiring! http://engineering.linkedin.com/ 44