Challenges surrounding Elastic Map Reduce and best practices with AWS.

1. BloomReach and AWS Elastic
MapReduce
Prateek Gupta – Lead Engineer
10/24/2014

2. The BloomReach
Personalized
Discovery
Platform
http://bloomreach.com/what-we-
do/

3. About BloomReach’s applications
Organic
Search
Contentunderstanding
What it does
Content optimization,
management and measureme
nt
Benefit
Enhanced discoverability and
customer acquisition in organic searc
h
What it does
Personalized onsite search an
d
navigation across devices
Benefit
Relevant and consistent onsite
experiences for new and known users
What it does
Merchandising tool that understa
nds products and identifies oppo
rtunities
Benefit
Prioritize and optimize
online merchandising
SNAP
Compas
s

4. BloomReach Organic Search -
Merchant Integration
Merchant domain Bloomreach domain
(Amazon Web
Services)
Cloudfront
domain: brcdn.combr-trk.js
pix.gif Elastic
Compute Cloud
domain: brsrvr.com
REST API
request
domain: brsrvr.com Elastic
Compute Cloud
Javascript
API response

5. BloomReach Organic Search
Architecture
API response
REST API request
Domain Name
Server (DNS)
AWS Load
balancer
Instance Instance Instance Instance
Alternate Cloud
Provider
Multiple Availability Zones
Domain request
Domain response

6. Example Workflow -
Personalization
Compute
User
Features
Compute
Recommen
dations
Compute
User Profile
User/ Product Database
Pixel
Logs
(S3)
Extract
Related
Users
Extract
User
Session

7. Elastic MapReduce (EMR)
Usage
• We serve 150+ customer websites
 100+ million pages processed/ day
 Users we see per day > 400M
 Multiple hadoop steps (clusters)
Usage Metric BloomReach Volume
Clusters per day 1500-2000
Hadoop jobs per day 5000-6000
Instance hours per day 25,000 – 30,000

8. Elastic MapReduce Usage
Growth
0
100000
200000
300000
400000
500000
600000
700000
800000
Q4
2009
Q1
2010
Q2
2010
Q3
2010
Q4
2010
Q1
2011
Q2
2011
Q3
2011
Q4
2011
Q1
2012
Q2
2012
Q3
2012
Q4
2012
Q1
2013
Q2
2013
Q3
2013
Q4
2013
Q1
2014
Q2
2014
Q3
2014
Spot
Instance
SNAP
Mobile
SNAP
Desktop
Compass
Instance
hours/
month
Organic

9. Challenges
• Cost containment
 On demand vs spot usage
• Cost tracking
 EMR tags
• Cluster setup delay
 Sharing clusters
• Cluster lifecycle management
 Terminate long-running clusters

10. Resource Selection
• Dynamic resource (instance type)
selection based on CPU, memory
maxCpuPerUnitPrice = 0
optimalInstanceType = null
For each instance_type in (Availability Zone, Region)
{
cpuPerUnitPrice =
instance.cpuCores/instance.spotPrice
if (maxCpuPerUnitPrice < cpuPerUnitPrice) {
optimalInstanceType = instance_type;
}
}

11. Workflow Management
• Makefile
• A framework for flow control using python
meta programming
A
C B
D
Valid Flows:
A->B->C->D
A->B->D->C

12. EMR Best Practices
• Use spot instances for cost optimization
• Use EMR tags for cost tracking
• Share EMR clusters for small jobs
• Keep track of long-running clusters
• Use optimal resource type based on
resource usage (e.g. CPU, memory)
• Workflow management

13. Thank You!
Prateek Gupta, Lead Engineer
prateek@bloomreach.com
www.bloomreach.com