Inevitably at the end of the year, a slew of gaming articles proclaim that a console has been the greatest of the year. According to Metacritic, the PS3 had the best games this year, with an average metacritic rating of 70.4%. The XBox360 could only manage a paltry 66.7%, so if you bought the entire reviewed PS3 and Xbox360 catalogue of 2008 you’d have exactly 3.7% more fun with the PS3 titles.

That does mean you’d have bought and played to completion 363 games, which is beyond the means of most people, so the comparison is practically meaningless. Looking deeper into the mountain of data from Metacritic, it is possible to see some interesting titbits. The results might surprise you.

The simplest comparison to make would be to look at the top games of each system, simply by looking at the metacritic data, and then find the average score for those games. I’ve not filtered these results at all, and this should be easily replicable by taking the top 20 scores from each system and getting mean averages from the top 10 and top 20 games respectively.

The Wii lags behind by a few percentage points and the PS3 and XBox360 are neck and neck. The results indicate that the PS3 edges the XBox360 in the top 10 comparisons but that is reversed in the top 20. Two points need to be made here. Firstly, the difference is so small, 0.1% and 0.4% that it’s really not enough to be able to make a compelling case for dominance. Secondly, the lists are populated by the same names, in the Top 20 PS3 and XBox360 games; there are 11 multiplatform titles, such as GRID, Fallout 3 and Soulcalibur 4.

So, do we need to look at exclusives? Certainly that’s the trend, as more and more sites try to differentiate between the systems. There is a logic to this, but the methodology seems to be quite sloppy, in that sites will choose which titles are classed as exclusive and which are included in the analysis.

The following graphs are based on the scores of the top exclusive titles for each system. The only qualifier for this list is if they are platform exclusive, be they full-price or download titles. By my reckoning a good game is a good game and it matters little if one costs less than the other.

The difference is even less pronounced in this case. The Wii is slightly less impressive in the Top 10, but when we extend the list to 20, there is only a minor difference between the three consoles. This graph seems counter intuitive, as the PS3 had the two highest scoring exclusives of the year in LittleBigPlanet (95) and MGS4 (94), but after this there next highest score was Resistance 2 (87). Compare this with the Xbox360 which had three 90+ games, Gears of War 2, Braid and Geometry Wars 2 and three games just under, Fable 2, Portal: Still Alive and Rez HD. Suffice to say, if you bought any of the systems, if you were hungry for exclusive content, you would have been well served by all three.

As an aside, if you included PC games in this comparison it would be the best of all the systems, with a top 10 score of 87.3% and a top 20 score of 85.4%. It is also favourably placed in the comparison of top games, so the PC gaming industry is dead; long live the PC gaming industry!

It seems the exclusive angle is a dead end. A direct comparison between the systems should do the trick, surely? It was difficult to find a list of games that were released on all three systems that didn’t extend way down into the depths of mediocrity, so I’ve split this section in two. First of all, the by now familiar tri-system breakdown.

This list is made up of Cross-Platform titles such as Guitar Hero World Tour, Call of Duty: World at War and Pro Evolution Soccer. Scores range from pretty good, Guitar Hero scored about 85, to pretty average, The Legend of Spyro: Dawn of the Dragon, with about 60.

Again there is little difference between the three systems. You could argue that the XBox360 has an advantage overall, but it is such a minor advantage, 1.0% and 1.6%, that you could hardly call it convincing. However, there are extreme differences between individual games. Pro Evolution Soccer 08 is clearly better on the Wii, with a 7% advantage over the next best version. Conversely, it has the worst version of Tomb Raider: Underworld, with a measly score of 68 compared to 76 for the PS3/XBox360 versions. Indeed the biggest variations are between the Wii versions and everything else. If there is any conclusion from this data it is that the Wii is clearly better at certain games than it’s more graphically muscular cousins, almost certainly because the play experience has been tailored to suit the Wii’s unique control mechanisms. Where this hasn’t been done, or has been done badly, the console suffers. Overall, if you were to buy one console and stick with it, you wouldn’t see an overall advantage or disadvantage if you bought a range of cross-platform games.

So, is there a difference between the PS3 and XBox360? They are clearly similar systems, utilising roughly analogous control methods with minor graphical differences. Using a slightly different list of games, because there are more games that are on just PS3 and XBox360 than all three consoles, we see the following:

To appreciate these results, the full list of games and scores is below:

Far from sorting the wheat from the chaff, this is a stunning example of how close the two systems are. Of the twenty scores, 10 are exactly the same, five are one point away, four are two points different and one, the largest difference, is a meagre three points. The average difference is only 0.3% in the top 10 and 0.1% in the top 20.

This article shows that the differences between the consoles are smaller than you might expect. Even given the differences between the Wii’s scores for some games, overall the consoles are remarkably similar in their scores for exclusives and cross-platform titles. To proclaim that any platform is better than another based on the quality of their games is erroneous. Clearly, all platforms have had a number of very good quality games, notably Grand Theft Auto 4, LittleBigPlanet, Gears of War 2 and Super Smash Bros Brawl. You might consider the following:

The Wii looks to be the most volatile console; it’s scores for cross-platform games tend to be quite different to the average score. This doesn’t mean that it’s any worse, as the average shows, it’s just has different strengths. If a game has been optimised for the unique control system, the Wii tends to do well. Equally, if the game is a straight port, because of the graphical capabilities of the Wii, it will compare unfavourably with the other two consoles.

The differences between the PS3 and XBox360 are even less pronounced. Their scores for Exclusives and Cross-Platform titles are virtually identical. Looking at the individual scores of the cross-platform titles that were released for the systems we see that they are also remarkably close. The claim that any of the systems are better than the other seems to be mistaken at best and malicious at worst.

Why use Metacritic?

Metacritic aggregates data, which means that it takes a large number of reviews from the Internet and compresses them into a very simple score out of 100. This means that you can be fairly certain that the scores you’re getting represent a large cross section of the opinion for each game. This compares to a single site comparison, which uses only data from it’s own website. Any bias towards a system found in an individual site is reduced by Metacritic due to the sheer number of reviews it looks at.

While Metacritic isn’t perfect, the way I’m using the data means that any errors that have crept into the scoring system will affect all systems and all results equally.