1. Hadoop Summit 2011 Online Content Optimization using Hadoop Shail Aditya shailg@yahoo-inc.com

3. Effectively and “pro-actively” learn from user interactions with content that are displayed to maximize our objectives

4. A new scientific discipline at the interface of

5. Large scale Machine Learning and Statistics

6. Multi-objective optimization in the presence of uncertainty

7. User understanding

9. Content Ranking Problems X Y Most Popular Most engaging overall based on objective metrics Most Popular + Per User History Rotate stories I’ve already seen Light Personalization More relevant to me based on my age, gender, location, and property usage Deep Personalization Most relevant to me based on my deep interests (entities, sources, categories, keywords) Related Items and Context-Sensitive Models Behavioral Affinity: People who did X, did Y Most engaging in this page/section/property/device/referral context? Layout Optimization Which modules/ad units should be shown to this user in this context? Revenue Optimization Voice and Business Rules Real-time Dashboard

10. Yahoo Frontpage Trending Now (Most popular) Today Module (Light personalization) Personal Assistant (Light Personalization) National News (Most Popular + User History bucket) Deals (most popular)

12. Search: Web, Vertical

13. Online advertising

14. …Item Inventory Articles, web page, ads, … Use an automated algorithm to select item(s) to show Get feedback (click, time spent,..) Refine the models Repeat (large number of times) Measure metric(s) of interest (Total clicks, Total revenue,…) Opportunity Users, queries, pages, …

15. Problem Characteristics : Today module Traffic obtained from a controlled randomized experiment Things to note: a) Short lifetimes b) temporal effects c) often breaking news story

16. Scale: Why use Hadoop? Million events per second (user view/click, content update) Hundreds of GB data collected and modeled per run Millions of items in pool Millions of user profiles Tens of thousands of Features (Content and/or User)

17. Data Flow Optimization Engine Content feed with biz rules Rules Engine Content Metadata Exploit ~99% Explore ~1% Near Real-time Feedback Real-time Insights Dashboard Optimized Module

18. How it happens ? Additional Content & User Feature Generation Feature Generation ITEM Model STORE: HBASE 5 min latency Request User Events Modeling Ranking B-Rules SLA 50 ms – 200 ms 5 – 30 min latency At time ‘t’ User ‘u’ (user attr: age, gen, loc) interacted withContent ‘id’ at Position ‘o’ Property/Site ‘p’ Section - s Module – m International - i’ STORE: PNUTS USER Model Content ‘id’ Has associated metadata ‘meta’ meta = {entity, keyword, geo, topic, category} Item Metadata

20. Technology Stack Ingest Analytics and Debugging

22. Hadoop processing via a collection of PIG UDFs

23. Different flows for modeling or stages assembled in PIG

24. OLR, Clustering, Affinity, Regression Models, Decompositions (Cholesky…)

25. Timeseries models (generally trends – extract of user activity on content)

26. Configuration based behavior for various stages of modeling

27. Type of Features to be generated

28. Type of joins to perform – User / Item / Feature

29. Input : DFS and/or HBase

31. Stores item related features

32. Stores ITEM x USER FEATURES model

33. Stores parameters about item like view count, click count, unique user count.

34. 10 of Millions of Items

35. Updated every 5 minutes

36. USER Model

37. Store USER x CONTENT FEATURES model for each individual user by either a Unique ID

38. Stores summarized user history – Essential for Modeling in terms of item decay

39. Millions of profiles

40. Updated every 5 to 30 minutes

41. TERM Model

42. Inverts the Item Table and stores statistics for the terms.

43. Used to find the trending features and provide baselines for user features

44. Millions of terms and hundreds of parameters tracked

46. Provides ability to control non-gridifyable solutions to be deployed easily

47. Have different scaling characteristics (E.g. Memory, CPU)

48. Provide gateway for accessing external data sources in M/R

49. Map and/or Reduce step interact with Edge Services using standard client

50. Examples

51. Categorization

52. Geo Tagging

54. Run complex queries for analysis

55. Easy to use interface

56. PM, Engineers, Research use this cluster to get near-real time insights

57. 10s of Modeling monitoring and Reporting queries every 5 minute

59. Made it simple to build different kind of science models

60. Point lookup using HBase has proven to be very useful

61. Modeling = Matrices

62. HBase provides a natural way to represent and access them

63. Edge Services

64. Have provided simplicity to whole stack

65. Management (Upgrades, Outage) has been easy

66. HIVE has provided us a great way for analyzing the results