Friday, September 7, 2007

The ESRB "Review Process"

A recent PC Magazine interview sheds some light on the ESRB rating process, which leaves a lot to be desired. While I am a firm proponent of industry self-regulation, I have often been puzzled at the ratings some games get. For example, when I saw that Smash Bros. Melee was rated "T for Teen," my immediate thought was "What next, Barbie's Mystic Horse Adventure 7 being rated M?" For those not familiar, while Smash Bros. is a "fighting" game, it's no more violent than a typical Saturday morning cartoon and contains no blood at all, whatsoever. In any event, now that the ESRB has stated how the games are "rated," it makes more sense.

To summarize the interview, game publishers send in a DVD of selected scenes and a lot of paperwork to get the game rated. In fact, the process is outlined in detail on the ESRB website. The point being that the ratings board never plays the games. Yes, you read that right. The people who rate video games do not play the game they are rating. It would be the equivalent of basing movie ratings on a form and a trailer. Context would be wholly absent.

I can see the logic the ESRB is using. First, playing the games would require a release candidate, which could delay the process. Second, it would take their "trained reviewers" much longer to play through the games in full than it would to review some paperwork and a DVD. Third, and finally, it's entirely possible that some, if not a large section, of the reviewers may not be able to complete the games at all. Moreover, the system they have going has rarely been faulted (see Hot Coffee).

On the other hand, I get the impression that ratings for media content are more accurate when the reviewer takes the content in context and on the whole, rather than seeing mere snippets. Perhaps the better approach is to have the ESRB hire "designated gamers," and have the reviewers watch the game being played for some period of time in addition to the forms and DVDs in order to contextualize the game. Perhaps then Smash Bros. Melee would have been rated a more appropriate E or E10+ rather than T. On the other hand, perhaps the powers that be would just assume most games be rated a tier higher than the content actually is, either to give parents more discretion or to insulate themselves from complaints. In any event, with the recent Manhunt 2 controversy, I expect that this issue will likely be blown well out of proportion by certain people in the media and politics. If anything, it creates a harsher rating system, not a weaker one.

[Thanks Jonathan!]

[EDIT: Reader Andrew Eisen, in the comments, points out: "Additionally, ESRB staff, including raters (time-permitting), play the final version of both hand-picked and randomly selected games to verify that all the materials provided by the game's publisher during the rating process were accurate and complete."

My thought is that, while true, and a new addition to the process, it still isn't for the purpose of actually rating the game, or putting elements in context. It's just like a double-check once the game is rated, and only occurs sometimes rather than on all games rated.]

7 comments:

Unknown said...

To be fair, the ESRB does occasionally play the game, a recent addition to its ratings process.

"Additionally, ESRB staff, including raters (time-permitting), play the final version of both hand-picked and randomly selected games to verify that all the materials provided by the game's publisher during the rating process were accurate and complete."


Andrew Eisen

Mark Methenitis said...

While true, and a new addition to the process, it still isn't for the purpose of actually rating the game, or putting elements in context. It's just like a double-check once the game is rated, and only occurs sometime.

As I said, though, if anything this makes ratings more likely to be higher than necessary. Depending on your point of view, that may not necessarily be a bad thing.

The Benny said...

It also sounds more like the eventual playing of the final version after the game has been rated (and possibly shipped) is to make sure the game doesn't deserve a higher rating, so it's more to check that they haven't been deceived rather than being likely to prompt a lower reclassification.

I can understand the logic of not playing a game in its entirety prior to rating purely because of the time/skill restraints that put games in a unique position. No movie is likely to run past three hours and you won't find people unable to reach the end or experience the whole thing due to a lack of ability. But as a gamer there is a huge difference personally between watching a trailer (even a somewhat lengthy gameplay movie) and actually playing it.

I imagine cost and effort are another factor. Even with Xbox Live and PSN making demos for console games far more accessible than ever before there are still developers who don't produce a demo before release because it involves bringing people away from the retail version, adding some amount of cost and possible delay to the release date (particularly for smaller studios). A similar demo sent to the ratings board wouldn't necessarily have to be as polished as a proper demo, but it would still take extra effort on the part of the developer.

Being in Europe I've just checked my case for Smash Brothers Melee (purchased as soon as it was released over here) to see how it was rated by the voluntary ELSPA group and it has an 11+ on it, essentially the same as a Teen rating. But a quick search on a few retail sites has it with a more sensible 3+ under the newer PEGI system, suggesting there was a rethink somewhere along the line.

Jabrwock said...

One fundamental problem with playing the game, is that unlike a movie where the reviewers merely watch, and can rewind, pause, and replay a scene ad infinitum to double check content that can change a rating, a game reviewer has a much harder time doing so.

There is also the problem of unlockable content, special side quests, and other areas of the game that require either great skill, or a certain combination of events to access. These are the perfect candidates for video review, as the developer can list them one after another, and the ESRB can play, pause, and review, without having had to spend hundreds of hours slogging through the game just to find the one scene.

Even cheat codes wouldn't help, because they still depend on the developer providing the reviewer with 'shortcuts', under the assumption that the developer didn't just provide cheat codes so the more extreme scenes would be purposefully skipped.

Every scenario still involves developer "honesty" when it comes to reporting content. Even having a developer on hand to walk the reviewer through the game still assumes the developer is showing the reviewer ALL the content.

There is also the problem of "pre-release" versions, which are usually what are submitted for rating. The company finishes up the racier content, and while they're still polishing up the bugs that keep you from actually getting to that content, they can submit that particular content for rating.

For example, perhaps a scene involves strippers pole dancing as you walk through a club. The club may not be complete, the walking may not work, and the interface may crash every few seconds, but a video of the stripper dancing animation can still be submitted for rating, even though that entire scene is incomplete.

jenifer said...

Wanna get good video games on most discounted prices then try shopping at Couponalbum.com and save while shopping.........!!

shannon said...

the ESRB kind of reminds me of the BCS in college football... good conceptually but in the end kind of more frustrating and useless than anything else. their ratings process is flawed to say the least, and i really doubt that that'll change. i'm wondering if a lot of it isn't personal "opinion" on behalf of the testers; to me the gun that the metroid dude totes around is safe for pretty much everyone, but to joe tester it might be M. who knows...

John said...

I was able to get MOGS4 for $50 by going to Keepcash