A recent PC Magazine interview sheds some light on the ESRB rating process, which leaves a lot to be desired. While I am a firm proponent of industry self-regulation, I have often been puzzled at the ratings some games get. For example, when I saw that Smash Bros. Melee was rated "T for Teen," my immediate thought was "What next, Barbie's Mystic Horse Adventure 7 being rated M?" For those not familiar, while Smash Bros. is a "fighting" game, it's no more violent than a typical Saturday morning cartoon and contains no blood at all, whatsoever. In any event, now that the ESRB has stated how the games are "rated," it makes more sense.
To summarize the interview, game publishers send in a DVD of selected scenes and a lot of paperwork to get the game rated. In fact, the process is outlined in detail on the ESRB website. The point being that the ratings board never plays the games. Yes, you read that right. The people who rate video games do not play the game they are rating. It would be the equivalent of basing movie ratings on a form and a trailer. Context would be wholly absent.
I can see the logic the ESRB is using. First, playing the games would require a release candidate, which could delay the process. Second, it would take their "trained reviewers" much longer to play through the games in full than it would to review some paperwork and a DVD. Third, and finally, it's entirely possible that some, if not a large section, of the reviewers may not be able to complete the games at all. Moreover, the system they have going has rarely been faulted (see Hot Coffee).
On the other hand, I get the impression that ratings for media content are more accurate when the reviewer takes the content in context and on the whole, rather than seeing mere snippets. Perhaps the better approach is to have the ESRB hire "designated gamers," and have the reviewers watch the game being played for some period of time in addition to the forms and DVDs in order to contextualize the game. Perhaps then Smash Bros. Melee would have been rated a more appropriate E or E10+ rather than T. On the other hand, perhaps the powers that be would just assume most games be rated a tier higher than the content actually is, either to give parents more discretion or to insulate themselves from complaints. In any event, with the recent Manhunt 2 controversy, I expect that this issue will likely be blown well out of proportion by certain people in the media and politics. If anything, it creates a harsher rating system, not a weaker one.
[EDIT: Reader Andrew Eisen, in the comments, points out: "Additionally, ESRB staff, including raters (time-permitting), play the final version of both hand-picked and randomly selected games to verify that all the materials provided by the game's publisher during the rating process were accurate and complete."
My thought is that, while true, and a new addition to the process, it still isn't for the purpose of actually rating the game, or putting elements in context. It's just like a double-check once the game is rated, and only occurs sometimes rather than on all games rated.]