Brainy Gamer

Let’s play a game together. We’ll call this game “Guess the Score.” Before we begin, let me warn you that I will win this game. Don’t get discouraged. In the end we both lose.

Here’s how to play: 

1. You are a real-world developer of a AAA video game. We’ll define “AAA” as a game with a relatively big budget, promoted by a relatively ambitious ad campaign, and reviewed by at least 50 print and online review sources. We’ll agree that relying on rickety adverbs like “relatively” is a necessary evil when characterizing titles across a wide universe of releases. 
   
   
2. Dream up an idea for a game. Any genre or platform. Original IP or sequel. Anything you like.
   
   
3. Invent a snappy title, some way-cool art, and a knock-em-dead blurb that describes how ridiculously awesome, immersive, game-changing, and paradigm-shifting your new game will be.
   
   
4. Go make your game. I will wait.
   
   
5. A week before release, invite me to your studio to preview your new game and offer feedback. As you begin your pitch, I will close my eyes, plug my ears, and sing “La-la-la-la-la” until your pitch ends. Then I will enter a trance-like state and utter a few incantations for dramatic effect. Finally I will deliver my verdict: “Your game is a Metacritic B+. Good work!” Then I’ll disappear in a puff of smoke. Or I’ll just get up and leave. 
   
   
6. One week later your AAA magnum opus will arrive on shelves, greeted by a Metacritic composite score of 87. “Why, that Abbott fellow is a genius,” one of your co-workers will exclaim. “Indeed,” you will concur, quietly conceding my victory in our little game.

How on earth did I do it? Can such prognostication be learned, like a dark art? Why, yes it can. The source of my wizardry: a little data gathering (with help from a student named Donovan Bisbee) mixed with the unholy alchemic power of Excel.

Actually, it’s no trick at all. I simply collected composite “Metascores” on 58 major games released in the last 24 months. Two Nintendo titles bookended the list: Metroid: Other M at the bottom (79), and Super Mario Galaxy 2 at the top (97). You can see my list here.

The average among all 58 games is 87.3 - a high number, but not a surprising one if you’re familiar with the inflated nature of game reviews. I won't go on about this. It is what it is.

The bigger surprise, although admittedly not a shocker, is the scant standard deviation of 4.5 (the real secret to winning my “Guess the Score” game). Drop only a few outliers on both ends - SMG 2, Uncharted 2, Portal 2, and Metroid: OM), and the variance from the mean among the 54 remaining games falls to below 4.

This probably isn’t earth-shattering news to anyone who pays attention to game reviews, but I do think the data lends credence to the idea that major releases, with few exceptions, aggregate inside a very narrow spectrum of scores. Publisher, platform, genre - none of these appear to matter (unless you’re Valve or Blizzard). 

The data suggests that if you’re developing a AAA game, you’re probably headed for a B+, an A- if you’re lucky, or a plain old B if you’re not. Don’t worry about a C. That just won’t happen. Unless your name is Duke.

I have all sorts of issues with Metacritic, and I’m not prone to wringing my hands over review scores. The problem here isn’t about numbers, but about differentiation among games - the deadliest threat facing the mainstream video game industry.

Put simply, too many games look like games we’ve already played. While we may find notable variety in the indie dev scene, who among us could sit through last month’s E3 press briefings without gagging on deja vu? We’re producing ever more of the same old stuff. Under such conditions, it’s hardly surprising to find dozens of games sitting bumper-to-bumper in a Metacritic traffic jam. Not even the reviewers can separate them.

Perhaps it’s useful to know that SMG 2 soared well above the average; and maybe a sub-par score for a Metroid game says something important. But when the vast majority of other major titles fall within the same small range of scores, numbers lose their meaning. 

I think many of us have a creeping suspicion that the industry we've relied on to give us Halos and Metal Gears and Marios has become a snake devouring its own tail. When meager differences among games become indiscernible to everyone but experts, there is trouble in paradise.