Better Ad, Offer and Content Targeting with Membase and Hadoop James Phillips, Membase Manu Mukerji, ShareThis Ben Jackson, AOL Learn more about Hadoop @ http://www.cloudera.com/hadoop/

1. Better Ad, Offer and Content Targeting with Membase and Hadoop ShareThis James Phillips, Co-founder, Membase ManuMukerji, Architect, ShareThis Ben Jackson, Chief Architect, Aol. Aol.

2. What is Membase?

3. Data Manager Pure Key-Value Store Built-in Memcached CP behavior (AP option) Cluster Manager Configuration manager Replication supervisor Rebalance orchestration NodeCode Manager Protocol extenders Real-time aggregation Index management Membase Server 3 Memcached protocol NodeCodeManager (in development) DataManager Cluster Manager

5. Peer-to-peer replication

7. Largest integrated sharing network We make sharing simple, engaging & valuable Powerful Social Analytics & Audience Monetization About ShareThis 450/momillion consumers ~850thousand sites 50+social channels

8. This is how it works Sharing Behavior Log Files Search Keywords Page Views HDFS Map/Reduce Content Analysis Taxonomy Ad Server User Membase 2

9. ShareThis Ad Products

10. Targeting at Aol.

11. Internet Conentent and Advertising Over 80 O&O internet brands Advertising.com 3rd party ad network Reach 90% of US Sells Ad Serving technology ADTECH About AOL 10

12. How AOL uses Hadoop Cookie Classification Events Reporting Cookie Profiling Model Building Scoring Database Supported LAMP Reporting Ad Servers Hadoop Cluster

13. Real Time Cookie Scoring? Cookie Classification Campaign Insights Ad Servers Events Node Code Membase /MPI Cluster Report Generation Cookie Scoring Flume Model Creation To HDFS Cookie Scoring Report Generation Reporting System No DB Hadoop Cluster

15. Stores months of historic data

16. Long running model building jobs

17. Supports research and development

18. Flume

19. A reliable and open source solution to streaming event data from multiple sources to multiple sinks.

20. Membase/MPI cluster

21. Stores recent data (a few weeks)

22. Allows data access on how a cookie scores for each model

23. MPI performs very fast in memory processing on data, given real time updates to cookie classification.The Components

25. All events stored keyed by user cookie

26. Impressions, clicks, …

27. Enriching data (known demographic and behavioral data)

28. Ideal but not yet available

29. Sets stored for each key

30. Atomic insert/delete

31. Membase Node Code is comingNode Code Next Year

32. MPI Parallel processing environment Used heavily in high performance scientific computing Parallel reductions outside of Hadoop MapReduce MPI and Membase on same cluster Local access to membase data for fast analysis Cookie Scoring and Reporting via MPI

34. (this is outside of the traditional memcached model)

35. Need to store the list of cookie keys in special index locations in Memcached

36. All MPI Nodes talk to all Memcached Nodes

37. Large communication overheadMemcache Cluster MPI Cluster MPI processing using traditional Memcached API

39. Currently only available via Python

40. MPI is traditionally C/C++

41. Each MPI instance talks to the local Membase daemon

42. No external communication needed

43. Process all data in that local instance

44. Future exploration neededThe fast way w/Membase streaming

45. Test the speed of the Membase implementation of Memcached How quickly can ad servers access data associated with a user cookie? Goal 5 millisecond latency for all requests Can Membase handle concurrent reads and writes? Testbed – 3 Memcached servers with replication factor 1. Results Pushed ~ 14 GB of data onto the servers Tried reading data while pushing up to 20k/sec keys Found <1 to 3 millisecond response times for queries, easily meeting our needs Membase POC: Part 1

47. Integrating MPI/Membase

48. Very fast processing of recent server data

49. Is this faster than similar processing on Hadoop?

50. Test Bed

51. 10 servers with 128 GB of ram each.

52. Flow

53. Push cookie profiles into Membase from HDFS

54. Waiting on node code for a fully realized streaming implementation.

55. Run simple aggregation job in MPI

56. Use the MPI for parallel reductions

57. Stream Data using TAP

58. Do most of the computation locally, doing the bare minimum in parallel reductionsMembase POC: Part 2 (ongoing)

59. Have Questions? James Phillips, james@membase.com Manu Mukerji, manu@sharethis.com Ben Jackson, ben.jackson@teamaol.com