Review Philosophy

When I started to put together reviews for games that I was playing, I took a fresh look at the review process. I've been reading game reviews for a long time, and have recognized that certain mechanics are more effective than others, and more conducive to the kind of information that I want out of a review. I identified a small number of philosophies that I wanted my reviews to exhibit. Foremost in my mind was the realization that game reviews do, in fact, exert a strong influence on the direction of the game industry. There a lot of things about the game industry and its direction that frustrate me, and I wanted to make sure that my reviews wouldn't disproportionately reward or encourage those things. Obviously, this website is just an insignificant grain of sand on the beach of the internet, and my reviews aren't likely to reward or encourage much of anything. That's irrelevant, as far as I'm concerned. I make it a point to not let my actions be dictated by the number of people that may or may not witness them. If I'm going to sprinkle thoughts into the global cauldron, the best I can do is ensure that they are representative of my beliefs. So without further ado, I'll get off this big soapbox and jump onto some smaller ones.

The following are most of the guiding principles that I use to write my reviews.

No Padding

Many reviews go to great lengths to restate a game's premise, its story, characters, etc.. More often than not, I think this is done just to increase the review's length. There are times when it's appropriate, if it's important to clarify or expound on some aspect of a game's premise because it directly affects some mechanism in the game. Generally, however, I don't think it should be part of a review. I don't read a review in order to get the game's manual reworded for me. If I'm interested in a game, I can go to fan sites, the developer's site, or as a last recourse, the publisher's site, and have the game's premise presented to me in a very nice manner. A review should tell me how the game plays. I suspect that the padding often occurs because reviewers have a word count goal, which is unfortunate. I don't have a word count goal. 

Technology != Gameplay

I'm as susceptible as the next person to the allure of cutting-edge graphics. But I also value my time, more so than almost every other commodity in my life. I resent games that eschew design in favor of eye candy, and I'll call them on it. In fact, the more games I play, the less tolerant I become for games with bad designs because I know just how good games can really be. Part of the problem lies with the scoring systems used by reviewers that implicitly prioritize the pure technical aspects of games. I therefore set out to identify a scoring system that was evenly balanced, allowing for a game to score well if it plays well, regardless of its development budget.

It's Just My Opinion

Gaming is like sex. Everybody likes it their own way. One of the things that is most disappointing about the direction that the games industry is taking is that niche gaming is being abandoned. This is extremely unfortunate because it is the capability for games to satisfy even the most unique whims of entertainment that makes them so powerful. But I digress. The point is that I like gaming my own particular way, just like everyone else. I understand that my appreciation of a game is a reflection of my own gaming preferences. So one of my goals in reviewing games it to embrace this concept, to consciously personalize the review, to communicate my opinion of the game in a way that exposes my gaming preferences. This makes it so much easier to not only be honest about my experience with a game, but also for someone reading the review to be able to interpret the review in a way that is valuable for them. It is in this aspect, I believe, where most reviews really fail.

Too often, I'll read a review that offers up essentially "I liked it, so you should, too." This is akin to saying, "I liked this pizza, so you should, too." What's missing is the fact that the pizza had anchovies on it. I think reviewers (myself included) want on some level for their reviews to reflect some quantitative process, as though they are taking scientific measurements of a game, using some metric that will weather the test of time. Then they can pretend that they're just reporting the results of the NIST Fun Spectrograph, and everyone will know exactly how fun the game is. As a result, reviewers tend to shirk away from explicitly expressing their own biases, their own cultural and circumstantial conditions, and especially their own gaming history, all of which are as much an ingredient in their experience of a game as the interface, or storyline, or rendering engine. With this in mind, I make a concious effort to do the exact opposite: to put my predilections front and center. And in the event that I might not do a thorough job, I'll at least include a reminder that anything I've said might not be worth beans to you. In the end, it's just my opinion, which is almost assuredly not identical to yours. I won't get my feelings hurt if you don't agree. Go write your own review.

Use the Whole Spectrum

I think that most collections of reviews aren't consistently objective in their scoring. Often, even when the text of a review is appropriately critical, the score doesn't reflect those sentiments. I wonder what is the point of having a scoring spectrum if you only use the top two-thirds of it, or as is often the case, just the top quarter. How many times have you read a review that states something like "the interface is an exercise in frustration" and then gives the game 3½ stars for interface, or a review that relates that the AI couldn't find its butt with a flashlight and a map, and then scores the AI at 80%? So one of the primary goals that I set for my review process was to devise a scoring system that was more conducive to objective, well-distributed, scores. If a game doesn't distinguish itself in any way, then it should score in the middle of the spectrum. In fact, that is the fundamental tenet of the scoring system that I created: everything starts at 0, in the middle of an (essentially) open-ended spectrum. Initially, I've chosen a fairly low-granular scale, with only 7 possible scores for any given category. For every category, the game's score will be proportional to the degree to which it distinguished itself in either a positive or negative way. I believe that this will allow scores to more accurately reflect a game's entertainment value in relation to other games (keeping in mind that it's all relative to my personal gaming preferences).

 

Score

I originally wanted to create a scoring system that had a cool acronym. Almost immediately I realized that pursuing a cool acronym was compromising the value of the scoring options. As stated above, I wanted a scoring spectrum that was coarsely granulated. Part of me was reluctant to do this, wishing instead for a system that would allow me to theoretically take every game ever made and use their hypothetical scores to uniquely differentiate each of them (this is that lust for scientific legitimacy). Technically, this is possible, but I ultimately decided that it had little value. I decided there was no value in that level of differentiation because of the point I made above: everyone has their own gaming preferences. It's not like I'm trying to find the atomic weight of games; it's more like I'm describing which ones taste better. Well everyone has different tastes. Some people actually like anchovies. Even more to the point, what tastes good depends on what kind of mood you're in, and that obviously changes all the time. Thus, despite the intriguing challenge of creating a finely granulated scoring system, I chose a more typical "categorized" system (like a 5 star system). I felt like there was an intuitive range, 7, and that it is enumerated something like the following: extremely bad, very bad, bad, mediocre, good, very good, extremely good. I was also inclined to think of this range as being somewhat nonlinear (it would be a rare occasion to put something in either of the terminal sections). It occurred to me that I might be able to identify the most appropriate name for each section in the spectrum and I accumulated a list of candidates for each section. I then realized that the list itself was valuable. It provided me with a means of choosing a different descriptor from review to review if I wanted. I could choose something that had an appropriate connotation or subtlety associated with it. The numerical score would help enforce objectivity, but the descriptors would allow another level of personalization.

Here are the 7 ranges in the scoring spectrum, and the current list of descriptors:

The scoring spectrum is half of the system, the other half consists of the scoring categories, those attributes of a game that will receive a score. I've chosen 12 categories. There's nothing magical or particularly special about the categories, in fact I suspect they will shift and change over time. Rather, I chose the categories based upon a rough approximation of my priorities. In order to address the problem described above about the implicit (sometimes explicit) over-emphasis on a game's raw technical prowess, I selected a sufficient number of decidedly nontechnical categories in order to amortize the effect of a game's technical features on the final score. Below is a list of the current categories and a short description of each:

1. Gameplay:

   What does the player actually DO in the game? Is it engaging? Is it consistent? Does the game provide the player with opportunities to be skillful or creative? Skillful can come in lots of different forms. Perhaps the player needs to train hand-eye coordination in a specific way, familiarizing themselves with the pseudo-physics and behavior of objects in the gameworld. The player might need to exercise clever problem-solving skills, or be creative with whatever levels of freedom the game provides. The game might give the player the opportunity to experiment with balancing the interactions of different simulations, or just discovering what those interactions are. I don't mean to imply that a game needs to be overly challenging in order to be engaging, either. As long as the expectations of the player are clear, and the player's influence on the game mechanics are consistent, a game *can* be entertaining. Perhaps a game's objective is to provide a high level of sensory immersion, but it sacrifices some player freedoms to do this. In and of itself, this doesn't imply bad gameplay. Maybe it works and maybe it doesn't. This is all very nebulous, not surprisingly. What is engaging and what isn't can't be predicted by mathematical formulae. As I've said several times now, these reviews are my interpretations. If I think it's good gameplay, it's good gameplay.

2. Immersion:

   This category is an attempt to measure the extent to which a game accomplishes its goals by drawing the player into its gameplay. A game doesn't require cutting-edge graphics, or 3D audio to be immersive, so I try very hard to differentiate this category from the other specifically technical categories. Clearly it will be easier to gauge immersiveness for some games than for others. For an RPG, this category will be particularly important, and I would be more critical in scoring this category than I would for, say, a pure strategy game. Thus, two games might have identical Immersion scores, but side-by-side one might be obviously more immersive than the other. As with the gameplay, the important thing to consider is the game's intent. Would you give a Jackie Chan movie a bad review because of poor dialogue?

3. Interface:

   Every game has to solve the transition from the gameworld to the physical world. The player is sitting down (most likely), looking at a screen, listening to speakers, pushing buttons. The game's interface is the means by which it communicates with the player, providing stimuli and information to the player, and the means by which the player exerts their will on the game, signaling input events to the game that need to be interpreted in context. The score in this category represents the degree to which the interface dissolved the barrier between the gameworld and the physical world.

4. Robustness:

   This is more than just bugs. Make no mistake, bugs are a big part of it, but other aspects of robustness are a game's performance requirements, its scalability, its immunity to platform idiosyncrasies. It's a sad testimony to the nature of games that it's impressive when a game still runs 5 years after you bought it. It's also important to note that I include non-hardware related bugs in this category. I'll also factor in whether or not a game accommodates consumer modification (if it's appropriate).

5. Indoctrination:

   To indoctrinate is to imbue with learning, to teach or instruct. A game's indoctrination is the method and means by which it both introduces and initiates the player into its gameplay and gameworld (every game has a gameworld, even if it's extremely simple and abstract, like Tetris). This is accomplished most often through the game's documentation. Thankfully, however, games are also doing a much better job of indoctrinating through tutorials and in-game guidance, as well. The score for this category is intended to encompass all of these methods. Similar to Immersion, indoctrination is more important to some games and genres than it is to others. If a game doesn't require significant documentation, it certainly shouldn't be punished for not having it. However, if a game doesn't provide appropriate instruction, of any form, then that's a failing. Furthermore, if a game goes above and beyond just providing the necessary nuts and bolts, and does so in an entertaining fashion, then it should be rewarded for that.

6. Single Player:

   This is where things start to get really subjective. Notice that this category (and the multiplayer categories that follow) are distinct from the Gameplay category. The Single Player category is intended to capture the experience of playing the game by yourself. To some extent, this boils down to "is it fun?" A game can score poorly in every other category and still be fun. The odds aren't good, but it's possible. A game can have a single player experience that is engaging enough that you'll tolerate a terrible interface, or suffer through arcane hardware incompatibilities (god knows I have) in order to play the game. Perhaps a game offers a story that is compelling enough that the single player experience is enjoyable despite many other flaws. This category is a chance for games to be rewarded for being more than the sum of their parts. It's also the category to which I attach a lot of significance. If you want, you can think of this category as the "reviewer's tilt" that is often used in reviews. Essentially, this score is my attempt to valuate the enjoyment I experienced playing the game by myself.

7. Cooperative Multiplayer:

   Like the Single Player category, this one measures the overall experience and entertainment value of playing the game, only here the focus is on playing cooperatively with other people. Some games (not enough!) provide the opportunity to play the game against AI with other people as your allies. Co-op play is one of my personal favorite kinds of gaming, and this category exists so that I can reward games that include co-op. Of course, a game can inlude co-op but do a poor job of it.

8. Competitive Multiplayer:

   Competitive MP is the lowest common denominator of multiplayer. It's the ability to play in a networked environment with other people, where every player is out for themselves, typically in a zero-sum situation. This category gauges the experience of playing a game in this way, if it's available.

9. Team Multiplayer:

   Team MP is a distinct form of multiplayer, blending co-op with competitive MP. For some games, this will be the single most important category. Many games are targeting team play as their primary, perhaps even only, objective. Team MP introduces lots of very interesting and difficult problems for game designers.

These last three categories are the ones that I consider purely technical categories. That doesn't mean that they're not important, or that they don't have an impact on a game's entertainment potential. Often, it's quite the opposite, but it's important to recognize when they should be emphasized and when they shouldn't.

1. Artificial Intelligence:
   

   AI is another very interesting aspect of gaming, and one to which I pay particular attention because I'm so fond of single player and co-op experiences. The AI can make or break a game. Easily. It can both ruin and magnify immersion. It can render balanced and innovative gameplay completely null and void. It can also enable gameplay that wouldn't otherwise be feasible or even worthwhile. Suffice to say that this category reflects the degree to which AI helped or hindered a game's objectives.

2. Graphics:

   Like AI, I can summarize this category by saying that it reflects the degree to which a game's visual representation helped or hindered its objectives. I want to reiterate my assertion that bad graphics do not make a bad game, nor do good graphics make a good game. I would think this is obvious, but apparently it's not. Graphics can hurt an otherwise good game, and can make an otherwise mundane game perhaps worth your while. I like to think of graphics like the language in a novel. If the writing is terrible, the story had better be really good for me to bother reading it. Conversely, it doesn't matter if the language is well-crafted, grammatically correct, and even poetic; if the story is boring then the novel isn't going to be very entertaining.
   
   [Ahh, but here's the catch! How do you know if the story is boring until after you've read it? Typically, you see the blurb on the book's jacket (i.e. screenshots) and think, "Hey, that's some good language." Then you buy the game ...er book, read it and find out the story is boring and unoriginal. Too late! You've already paid for it. It's so much easier to make good language than it is good stories. This is one of the not-so-secret secrets of game developers and publishers.]

3. Audio:

   Audio is such an under-appreciated facet of games. Usually, if you notice the audio, it's because it's either extremely bad or extremely good. Nevertheless, it's a distinct part of a game's impression and can really help a game achieve its goals if done right. Like a number of the categories listed here, the audio is significantly more important to some games and genres than for others. I want to point out that I include both music and sound effects in this category. If a marked contrast exists between the two for a given game, I will try to comment on it specifically in the text of a review.

Note that if a particular category is not relevant for a given game, then the game does not receive a score in that category (denoted by "N/A" for "not applicable"). If a category is relevant to a game but for whatever reason I didn't feel it was appropriate to give a score, then again that category is not reflected in the final score (it will be denoted by "DNR" for "did not rate"). Included with the categorized scores are two total scores. The first one is a cumulative score that only factors in the potential (positive and negative) score based on the categories that were rated. Below that is the normalized score, which adjusts the total score onto a scale of [-100 to 100].  Why not normalize it to 0->100?  Because I wanted to reinforce the notion that the scoring is bi-polar, i.e. that there is a baseline (0), and that games can score worse than or better than that baseline.