Measure twice, cut once, weep daily

Remember when you could just buy something? You handed over money in exchange for goods or services and that was it.

But now we are pursued forever by the furies of online shopping, service metrics.

And so it goes on and on. The comforting notion that all this data has value, that the overhead in collecting, analysing and distributing the data is worthwhile.

That somehow these tick boxes, these questionnaires can all equal user testing, proper analysis, true customer engagement.

And yet all that happens is that we get the most superficial view of a landscape where the responses are from a small percentage of customers who are either angry, bored or just making stuff up.

And we call that “Business Intelligence”.

I was reminded of the limits of business intelligence this week when playing Destiny 2.

Destiny is a multi-player game set in the solar system where you fight aliens, take on fellow players in battles and complete quests.

Destiny has a very V shaped player distribution. At one end you have people who spend hours in the game, who play with others as part of a clan, and who are as much about PvP as PvE. At the other end you have players like me who play for a few hours a week at most and who play solo focusing on the quests and plot.

So if you are Bungie, the makers of Destiny 2, how do you balance a game for a V shaped audience? Your metrics tell you that half of the players completed the weekly story quest in less than 30 minutes.

So this week you put the quest behind a higher level of difficulty. “Defeat 50 Champions!” you say. The problem is that this does nothing to slow down the skilled players yet locks out the solo players like me. What happens if you bar 50% of your users from your service? Will they stay around?

Too many businesses do not understand what the shape of their user distribution is. Do you?