Impact AI 2020: Portfolio-Scale Data Science at Zynga

1. Portfolio-Scale Data
Science At Zynga
1
Ben Weber
February 4, 2020

2. 2
About Me
Distinguished Data Scientist at Zynga
@bgweber on Twitter, Medium & Github

3. 3
Applied Science at Zynga
Goal: Build Portfolio-Scale Data Products
Example Projects
• Propensity Models
• Recommendation Systems
• Player Segmentation
• Anomaly Detection
Tech Stack: AWS, Python, Databricks, CouchBase

4. 4
Zynga Games
4

5. Our Challenge
• We have tens of millions of players and dozens of
games across multiple platforms
• Our games have diverse event taxonomies
• We want to build accurate models for personalizing
our gameplay experiences
5

6. “One of the holy grails of machine learning is
to automate more and more of the feature
engineering process.”
Pedro Domingos
CACM 2012
6

7. Our Approach
• Leverage ML libraries to automate feature engineering
• Develop Portfolio-Scale data products
• Empower our game studios with ML models
7

8. Use Cases
8

9. Applications
Propensity Models: What actions are players performing?
Segmentation: Who are our players?
Anomaly Detection: which players are bad actors?
Recommendation: What actions should they take?
9

10. Feature Encoding
Input Dataset
• Thousands of events per player
Feature Generation
• Aggregation with FeatureTools
Output Dataset
• A single row per player
10
Raw
Event
Data
Player Summaries

11. Propensity Models
• We predict which users are likely to act using classification models
• Game studios use propensity scores to define experiment groups
• Feature generation reduces the need for manual feature engineering
11
Data
Extract
Feature
Engineering
Feature
Application
Model
Training
Model
Publish

12. Segmentation
• Generated features are used as input to k-means clustering
• Archetype labels are assigned based on qualitative analysis
12

13. Anomaly Detection
• Players are represented as 1D images
• We train an autoencoder to reduce dimensionality
• Players with large vector differences are flagged as suspect
13
Features
Latent
Space
InputLayer
OutputLayer
Players
Features
Players
AutoencoderInput Vectors Output Vectors

14. Recommendation Systems
• Feature engineering is used for item & guild recommendations
• Cosine similarity is applied to normalized generated features
Item Recommendations
sim(u, v) = u * v
|| u || * || v ||
weighti
= ∑ sim(u, w) * rating(w, i)
w = user neighborhood
14

15. Feature
Engineering
15

16. FeatureTools
• A python library for deep feature synthesis
• Represents data as entity sets
• Identifies feature descriptors for transforming your
data into a shallow and wide format
• Open-source version maintained by FeatureLabs
16

17. Kaggle NHL Dataset
17

18. 18
Data Frames
game_df
plays_df

19. 19
Entity Sets
• Define the tables and
relationships for DFS
• Operate on Pandas
data frames

20. 20
1-Hot Encoding

21. 21
Deep Feature Synthesis

22. Applying FeatureTools
• We translate our raw tracking events into player summaries
• Supports dozens of games with diverse taxonomies
• Minimizes manual steps in our data science workflows
• Scales to millions of players and billions of records
22

23. Deployment
23

24. Tech Stack
• Databricks for PySpark
• FeatureTools for generation
• Pandas UDFs for distribution
• MLlib for predictive modeling
24

25. • Introduced in Spark 2.3
• Provide Scalar and Grouped map operations
• Partitioned using a groupby clause
• Enable distributing code that uses Pandas
25
Pandas UDFs

26. 26
UDF
Pandas
Output
Pandas
Input
Spark Output
Spark Input
UDF
Pandas
Output
Pandas
Input
UDF
Pandas
Output
Pandas
Input
UDF
Pandas
Output
Pandas
Input
UDF
Pandas
Output
Pandas
Input
Grouped MAP UDFs

27. 27
Feature Generation at Scale

28. AutoModel System
•Generates hundreds of propensity models
•Powers features in our games & live services
28
Data
Extract
Feature
Engineering
Feature
Application
Model
Training
Model
Publish

29. Wrapping Up
29

30. Data Science at Zynga
Old Approach
• Custom data science and
engineering work per model
• Months-long development cycles
• Ad-hoc process for deploying
models to production
30
New Approach
• Minimal effort spent on the
feature engineering stage
• No custom work for new games
• Model outputs are published to
application databases

31. Takeaways
• Zynga is leveraging automated feature engineering to build
Portfolio-Scale data products
• We are using PySpark to scale to tens of millions of players
• Feature generation has unlocked novel data products
31

32. Looking Forward
• Reinforcement Learning
• Real-time personalization
• Procedural Content Generation
32

33. 33
Portfolio-Scale Data Science At Zynga
Ben Weber
Distinguished Data Scientist
bweber@zynga.com
https://www.zynga.com/jobs/