Trends in game analytics: What’s happening and why? By Heather Stark, Analyst at Kinran Limited.
The Games Industry Analytics Forum returned for its 10th meet-up on Thursday 27th August at Product Madness in London.
GIAF is a free event for game analytics practitioners held in both the USA and UK, organised by game analytics & marketing company deltaDNA.
Featuring ever-changing presentations, venues and expert panel discussions, it's a unique opportunity for practitioners looking to generate insight and value from big data game analytics; one of the most important trends in games.
Interested in speaking at a future event or in finding our more? Visit www.deltadna.com/GIAF
8. Kinran
HAStark
...[according to VB research] most mobile-
first companies are trying to pay between
$1 and $1.50 for users, but they are only
getting quality users at multiples of those
numbers...
August 12 2015
http://venturebeat.com/2015/08/12/this-service-tells-you-what-supercell-machine-zone-and-other-big-
publishers-spend-on-user-acquisition/
9. Kinran
HAStark
...[according to AppScotch] Machine Zone
is currently spending somewhere around
$12 per user with AdColony, InMobi, and
Unity Ads, up to $20 per user with Vungle,
and between $2 and $30 per user with
Chartboost...
August 12 2015
http://venturebeat.com/2015/08/12/this-service-tells-you-what-supercell-machine-zone-and-other-big-
publishers-spend-on-user-acquisition/
15. Kinran
HAStark
Kevin Schmidt and Luis Vicente, Mind Candy
Practical real-time approximations using Spark
Streaming
hyperloglogs to count uniques
bloom filters to count revenue
stream-summary for top-k
(metwally agrawal abbadi 2005)
from nucl.ai Data Science track (to be published)
earlier version from huguk available now
http://www.slideshare.net/huguk/fast-perfect-practical-realtime-approximations-using-spark-streaming
21. Kinran
HAStark
Miloš Milošević, Nordeus
Early Churn Prediction and Personalised Interventions In Top 11
later detection is more accurate - but less useful
tried many techniques – logistic regression good!
cluster users based on first day gameplay
customise messaging based on clusters
increased D1 retention (and downstream metrics)
from nucl.ai 2015 Data Science track (to be published)
writeup available on gamasutra now:
http://www.gamasutra.com/blogs/MilosMilosevic/20150811/250913/How_data_scientists_slashed_early_churn_in_Top_Eleven.
php
24. Kinran
HAStark
Meta S. Brown
Analytics failure and how to avoid it
Analytics programs fail...
.... because they lack a viable plan for success
Imperial College Data Science Institute 24 June 2015
http://www.slideshare.net/metabrown/analytics-failure-how-to-avoid-it
25. Kinran
HAStark
Meta S. Brown
Analytics failure and how to avoid it
Analytics programs fail...
.... because they lack a viable plan for success
Define success, and who decides on it
Imperial College Data Science Institute 24 June 2015
http://www.slideshare.net/metabrown/analytics-failure-how-to-avoid-it
26. Kinran
HAStark
Meta S. Brown
Analytics failure and how to avoid it
Start with a business problem
a small one
understand the business problem really well
As you scale up – pay attention to process
replicable! replicable! replicable!
Imperial College Data Science Institute 24 June 2015
http://www.slideshare.net/metabrown/analytics-failure-how-to-avoid-it
27. Kinran
HAStark
Meta S. Brown
Analytics failure and how to avoid it
Use only as much data as you need to
The best use case for Big is personalisation
Imperial College Data Science Institute 24 June 2015
http://www.slideshare.net/metabrown/analytics-failure-how-to-avoid-it
29. JOIN IN THE CONVERSATION PARTICIPATE IN THE NEXT GIAF
Analytics for Games
www.deltadna.com/giaf
events@deltadna.com
#UKGIAF
Notas do Editor
I’m going to take a look at what’s going on in game analytics. And why.
I’m going to look at four aspects – what’s going on in the market we serve – in the games industry itself – and how that’s shaping what gets done in analytics.
And I’m going to look at changes in the tools and tech we have at our disposal, to do the work we do.
I’m going to look at frontier zones – areas what gets done is evolving. Liminal zones where the ocean and fog meet.
Finally, I’m going to look at failure prevention in games analytics. (I didn’t want to leave you sad by titling it failure.) This is a glass half full take on things.
I’ve only picked out a few things under each heading, and there might be more that you think of. Tell me in question time, or afterwards in the bar.
You’re probably wondering. How does she know all this stuff? Has she wired my studio up for telemetry? Or is she making it all up?
Well it’s neither. I do what you do – talk to people, read trade press, listen in on social media, look at changes in vendors’ service offerings, go to events, and use this info as tell-tales to see which way the wind is blowing.
Last month I curated and chaired the Data Science and Analytics track at the games AI conference in Vienna, and I’m bringing back a few shiny snippets from that to share with you. Apparently crows don’t actually like shiny things, which I found out after researching this cool picture, but I do.
It makes sense that what gets done in games analytics would be influenced by what’s going on in the games industry itself. So what’s going on in games? It depends a lot on what facet of the market you live in.
The market as a whole is still growing. Could be worse.
But there are two trends that make life difficult. They have to do with distribution – both on mobile appstores and on steam. One trend is the increasing number of games on the market. The other is the stickiness of the top 10 lists.
Visibility in the face of competition, both long-tail and top 10, is a huge challenge.
Best sellers tend to hang about like low cloud over England in summer. This isn’t by accident. It’s to do with store managers wanting to optimise their returns, and giving successful titles visibility via multiple internal channels. Also also, on mobile appstores, it’s about sophisticated use of the advertising ecosystem by top sellers.
Also - the game needs to be good. But that isn’t enough.
What’s happening as a result? It’s making people pay even more serious attention to distribution and visibility. Some are choosing to go with publishers rather than self-publish.
Ouch.
Really. Ouch. It’s bleeping expensive. This isn’t the kind of spend to do casually.
This means having a good grip on your i/o for acquisition. This can get complicated. But the key point is that different players come in from different sources, which have different costs. You need to balance that view of your costs, with predictions about likely revenue. These predictions will become more accurate the longer a run of real data you have, but by that time your media buying window may have closed.
Since it’s so nailbitingly pricey to acquire players, there is an increasing focus on understanding how to keep them. This has always been of interest. But the truism that usually cheaper to retain a customer than to get a new one is being taken more seriously, now that competition for attention is fiercer than ever.
The areas that service providers are focussing on is often directional. App Annie has recently begun to offer competitor intelligence on how players interact with other games. After offering to integrating player metrics with their store performance data, for free.
There’s lots to say about tech enablers. I’m only going to give a light kick to two aspects of it here – but ask me other things in the question time, or afterwards.
One thing I’m seeing is the need for speed, for taking certain decisions. And with that an interest in streaming architectures and algorithms, particularly Spark streaming. There’s a good piece of work from nucl.ai on this, from one of our London games firms.
I’d like to give a shout out to some open sourced work by Mind Candy, on using probabilistic data structures for stream-based metrics. This is very like the material covered in Ilya Katsov’s ‘Highly scalable’ blog, but it includes links to the source code. Most of these approaches use hashes to enable constant-space scalability, at the expense of perfect accuracy. Agrawal from Berkeley is the author to watch here, if this is your bag. He’s got a whole book on it.
Also on the tech enablers front, there’s something almost unbearably hot in the machining learning world, that hasn’t yet become standard operating procedure in game analytics: deep learning.
Actually this is a better picture. There’s huge excitement about deep learning as it enables the system to learn the features which are important – and not only that – learn a hierarchy of features, with lower level features being more general, and higher level features being category specific. This has resulted in big progress in speech recognition, and visual processing. The visual processing work is particularly interesting as it dovetails well with work on neurology of visual processing, and on mathematical modelling of processing channels.
I have heard of people using it in analysis of play data, for player segmentation, but it’s been more along the lines of ‘I’ve tried everything but the kitchen sink and here’s this cool thing I will try too - I’m not quite sure what it’s good for but hey why not’.
I’d say adoption is at the garage tinkering stage. Hence the picture of the messy machine shop. Which looks a bit like my desk.
I’m not sure what problems in game analystics are the right shape, and have a strong analogy to signal processing. I think that’s tbd.
By frontier zone, I mean places where practice is evolving, interesting, and not settled. You could I guess count the use of deep learning as a discovery technique as a frontier zone. Here are a few more.
I’m not talking about Dark Side of the Moon, as the final frontier. But about being able to segment players (and game elements) in a way that informs design. There’s a particularly good piece of work in this direction from nucl.ai
There has been a fair amount published on churn prediction detection – game play gives off a number of signals that can be useful in this respect. There was some interesting work at CIG2014 on that last summer, from Wooga in collaboration with University of Lausanne. What there’s been less of, is work that integrates churn detection with churn prevention.
Here’s a really good piece of work from the track I chaired. What’s good about it combines analysis for prediction, for analysis to guide interventions. And it worked really well for them.
Another frontier zone I’d wave a hand to is a meeting of minds between games user research and quantitive product management. Games user research is a discipline with its own set of conferences. I went to one in July, in London, and there’s going to be a big one, ChiPlay in London in October. In the whole day there was only one talk that had any numbers in it.
It may not be obvious but these guys could make a great music together if they’d only learn each other’s languages.
Finally here’s something everyone cares about. My poster child for this once again comes from someone else’s work: Meta Brown. She wrote Data Mining for Dummies. She’s not a dummy though. She has an advanced degree in nuclear engineering from MIT. She’s done lots of big ticket analytics project work, and consulting. She gave a talk earlier this summer at the Imperial College Data Science Institute. I’ve never laughed so hard in a data science talk. I can’t begin to imitate her dry midwestern sense of humour, but there was some good stuff she said that I think bears repetition.
This is like the kind of koan you can chant while walking around whacking yourself on the head with a board. It’s that good.
If you don’t get clear agreement about what success looks like – and to whom – success is going to be elusive. Hugely so.
I’m not talking here about collecting basic metrics, but about going offroad, or on a deep dive in search of treasure.
The key message here is to agree the business problem – and understand it as well as possible – before touching a drop of data. I do see people enjoying just diving in to see what’s there, and that’s fun for a side proejct, but being prepped property makes it easier to explore further.
And as you get more complicated – you need more process support. That means Crisp-DM, or some cousin. According to Meta.
Don’t go mad for big data when small data will do. Use it for what it’s good for.
Here I’ve talked about what I think’s going on at the moment. As to what next, let’s take it to the bar.
