Game Analytics I: DON’T Measure Everything

It’s no secret that turning a profit in F2P mobile games isn’t easy these days, and not only due to fierce competition. The modern mobile game service is an intimidating construct with many moving parts – branched code, staged builds, content production, backend, 3rd-party integrations (and updates), device fragmentation, marketing and distribution, just to name a few. When managing such a product, it’s easy to be swept into the hurricane of inputs and lose sight of the big picture: where we are, where we’re going, and what most deserves our focus right now.

At Turbine Games Consulting, we very frequently hear variants of the same basic question: “How should we (better) use our data to make 80/20 decisions?” For example:

  • “What should we be focusing on?”
  • “We’re collecting a lot of data.  Now what?”
  • “How can we develop in-house capability around analytics and data-driven decisionmaking?
  • “How can we find small adjustments to our game that yield large returns?” *(our favorite question!

These questions implicitly reflect the challenge of making analytics, statistics and F2P game business models straightforward, easy to understand and actionable, in a competitive industry that often incentivizes doing the opposite.

One of the most important lessons that I’ve learned (and relearned) over the years is the miraculous power of keeping things as simple as possible. This principle has countless applications, but has become fundamental to how I approach mobile game design, analytics, live operations and ARPU-hacking

After working as a Product Manager on over 130 games, and making plenty of mistakes, I’ve found that a common-sense, no-frills approach to game telemetry yields the greatest returns in 80-90% of situations. As enticing as it sounds to track everything that users do within a game, I’ve found that, for a variety of reasons, the real world inevitably punishes us for our attempts to do so:

  • Bugs: All other things equal, tracking more actions leaves less time to implement each. This tends to increase bug counts.
  • Loss of trust in data: After encountering a few bugs, you won’t have 100% confidence in the data, which in turn means you can’t fully trust your results and conclusions. You’ll find yourself cross-checking with other data, only to find another data point that seems off, and down the rabbit hole you go…
  • Visual and mental clutter: You may find yourself routinely wading through long lists of superfluous metrics to reach the truly important ones that you use 95% of the time.
  • Not a substitute for careful planning: Measuring everything is a tempting but poor substitute for the careful design that your events taxonomy requires.
  • Adds noise: Superfluous metrics produce a fog of ‘interesting observations’ that distract from your team’s focus on the most important and actionable metrics.
  • Makes you a bottleneck: Overly-complex telemetry makes it harder for others on your team to self-serve and do independent analysis.

For these reasons and others, I prefer to start by establishing a rock-solid foundation that relies on only tracking six critical user behaviors. The next post explores this topic in further detail.

The articles that follow will explore – in more depth – how to apply a simple, common-sense approach to mobile game telemetry and data-driven decisionmaking. Of course, there’s no one right approach. Every mobile business has a unique perspective and toolkit, and my favorite part of consulting is the learning and knowledge-sharing that comes from these experiences.

The next article, ‘Just Track These 6 Behaviors,’ dives into tactical analytics design, and will be most relevant to mobile product managers already familiar with the basics of mobile game telemetry.

But – no matter who you are – I’d love to hear your thoughts and perspective. Don’t hesitate to contact me at [email protected]

About the Author