Early Churn Prediction and Personalised Interventions in Top Eleven game

1. Early Churn Prediction and Personalised Interventions in TOP 11 Miloš Milošević, Nordeus
2. About me Data engineer at Nordeus. Main focus: Predictive machine learning pipelines; Their deployment and maintenance.
3. About Nordeus ● Award winning gaming company ● Offices in London, Dublin, San Francisco, Skopje and Belgrade ● Creators of Top Eleven
4. About TopEleven ● Our flagship product ● 110+ million registered users ● Global ● 5 years old
5. Agenda IDENTIFY - Predict early churners using machine learning TARGET - Construct a personalized message for every churner INTERVENE - Send messages via a scalable notification system
6. Early churn ● What is it? ● Why is its reduction important? ● Can we predict it and how?
7. When to engage the prediction algorithm?
8. We tried 3 things
9. Features
10. Models ● Best model - Gradient Boosting Trees ● Model we use - Logistic Regression ● Model to play with - Recurrent Neural Network
11. And what do we do then? ● Optimize backend ● Target churners ● Stay prediction
12. Targeting Messages need to be meaningful, otherwise...
13. What kind of notifications work well on what type of user? How to find the perfect match? Use the user’s first day activity to create meaningful personalized notifications. Targeting
14. User segmentation
15. Examples
16. Measuring results Split users into 3 groups: ● Control ● Baseline ● Test
17. Our results ● On average, baseline group achieved 30% better retention ● Test group achieved 40% better retention than baseline
18. Notification engine Specs: ● Modular design ● Scalable ● On time delivery ● Near real time message creation ● Logging system ● A/B test support
19. The power of Spark
20. Conclusion ● Everyone can do early churn prediction ● Target with care ● When in doubt, Apache Spark
21. Questions? milosmi@nordeus.com @mmilosevic10

Notas do Editor

Hi everyone and thanks for coming. My name is Milos Milosevic, I am from Nordeus and today I’m gonna talk to you about early churn prediction and personalized interventions in TopEleven
So, first off, a little bit about me. I am a data engineer at Nordeus and my main focus lies in predictive machine learning pipelines, their deployment and maintenance in a live production setting.
Regarding Nordeus, we are an award winning gaming company, founded in 2010. We have offices in London, Dublin, San Francisco, Skoplje and Belgrade.
Also, we are the creators of Top Eleven.
In case you aren’t familiar with TopEleven, it is a social online cross platform free to play football manager game.
When you register, you are given a football team, you can play around with your team tactics, train your players, sell and buy new ones and of course play matches.
You have 3 competitions per season, league, champions league and cup, while the season itself lasts 28 days.

TopEleven is our flagship product. It is 5 years old, and global, with over 110 million registered users from every country in the world playing it.
So, I am here today to talk to you about how to solve the problem of early churn.
In order to do that, we will discuss how to identify and predict early churners using machine learning models.
Furthermore, we will see how to target the churners and construct personalized messages that maximize reactivation.
And in the end, we will touch upon the underlying infrastructure needed to reach out to these users and actually send these messages.
Okay, so, can you tell me, what is early churn?
Churn alone, it is a widely known term in many industries, including banking, telecommunications and gaming.
When people think about churn, it is mostly about veteran or late churn, however, nobody ever speaks about early churn.

In gaming, there can be many definitions of early churn, and they vary based on the game, but some general definition is that,
early churn is when a user comes into your game, plays the game for a limited short period of time, and then leaves the game, never to return again.

If we reduce early churn, we increase retention, and by increasing retention we can increase all other metrics. Think about it like this, if a user is to stay longer in the game, this increases the chances that the user invites his friends to the game or spends some money in the game.

Now, early churn is mainly addressed in a game by creating a good tutorial, or by optimizing the first few levels of the game. If you would want something about it further, normally you would need to create and developed a new in game feature. Depending on your game, and the magnitude of the feature, actual implementation of it can take ages.

But, what if there’s something, that we, as analytics people, can do, to reduce early churn, without burdening the product team with new features to create.
This was exactly the idea we at Nordeus had that sparked our effort towards firstly identifying early churners. So we sat down and thought about how can we create a model that predicts this. And this led us to interesting question that we haven’t thought of before, and that question is ...
… when to engage the prediction algorithm?

So, this is a very important question and depending on your definition of early churn, the answer can vary. But also, depending on your particular use case for early churn prediction you may want to engage the algorithm sooner or later.

When we started out, we knew we wanted to do targeting, but we still couldn’t answer this question. So what do you do when you’re not sure about something? You test.
We made 3 slightly different models, predicting after 3 days, 2 days and 1 day after user registration, and here are the results.

Early Churn Prediction and Personalised Interventions in Top Eleven game

Esté a ler uma amostra.

Confira estes a seguir

Early Churn Prediction and Personalised Interventions in Top Eleven game

Mais Conteúdo rRelacionado

Semelhante a Early Churn Prediction and Personalised Interventions in Top Eleven game (20)

Mais recentes (20)

Early Churn Prediction and Personalised Interventions in Top Eleven game

Notas do Editor

Early Churn Prediction and Personalised Interventions in Top Eleven game

Esté a ler uma amostra.

Early Churn Prediction and Personalised Interventions in Top Eleven game

Recomendadas

Mais Conteúdo rRelacionado

Semelhante a Early Churn Prediction and Personalised Interventions in Top Eleven game (20)

Mais recentes (20)

Early Churn Prediction and Personalised Interventions in Top Eleven game

Notas do Editor