I've received a few emails asking why I've started tracking Variety's game reviews, reviews which are not published with scores. I have a couple of thoughts. I've been talking to Variety's Games Editor for over a year now, and have discussed our potential working relationship with their Editor-in-Chief on a couple of occasions. The Variety writing staff is made up of industry veterans who are extremely professional and who take their jobs seriously. I've vetted them and would not have picked them up had it been otherwise.

In other sections of Metacritic, we regularly track publications that do not assign a score to their own reviews (LA Times, NY Times, Hollywood Reporter, Variety), and we've always estimated the scores based on the impression gleaned from the review. I've only very rarely taken this approach in the Games section of Metacritic because gamers and the games industry are so sensitive to our scoring system and process. However, I did track the New York Times game reviews for years - a site which had no scores. In that case, the lead critic from the Times, Charles Herold, emailed me his unpublished scores every time the NYT published a review and I used those numbers on Metacritic. (See the scoreless New York Times review of The Orange Box by Charles Herold which appears on Metacritic's Orange Box page with an 88 score) My argument is that if the score comes from the horse's mouth, from the critic him or herself, so that I know the critic's intent definitively, I am satisfied that I can maintain 100% accuracy. (Incidentally, we stopped tracking the NY Times when they laid off their veteran lead critic a few months back.)

So with respect to Variety, I've come to the same arrangement with their lead editor. Each time a game is reviewed, he sends me the link to the review along with their unpublished 0 - 100 score - a score which I post at Metacritic without alteration or personal input. I've vetted a great many reviews and scores from Variety's team before deciding to pick them up like I do with every website and magazine that I decide to track.

And a further note: I receive hundreds of requests from publications asking to be included in Metacritic's process every year. I take each of them seriously and evaluate them using a host of metrics and standards. I've been doing this since 1999, and I'd doubt that anyone has read many more game reviews than I have over the same period. I would suggest that I'm competent to make such decisions, and I am the last and only word on which publications make it to Metacritic. So if you're disappointed with any publication's review of a particular game, step away from your conspiracy theories and know that I've always been the custodian of Metacritic Games and I take this process extremely seriously. Nobody on my team at Metacritic or our parent companies has ever asked me to add or drop a particular publication or individual reviews. They've also had no part in assigning weights to the individual publications. I alone handle this process.

If you have any further questions about my process, I'd be happy to field them via email.

As always, I thank you for using the site.

Best,

Marc Doyle
co-founder, games editor
metacritic.com

This message has been edited. Last edited by: Metacritic Games Editor,

That's fine Marc, the problem is look at Variety's reviews of 360 games verses ps3 games. Variety is giving ps3 multiplatform games low scores (DeadSpace), while posting high scores for 360 mutliplats (Fallout 3). For exclusives, Variety has given Fable 2 one of its highest marks, while giving Resistance 2 and LittlebigPlanet for ps3 their 2 lowest scores of any review source on the internet. This is lowering the Meta score for ps3 games while raising scores for 360 games. You may have vetted these writers but that does not mean the overall effect of adding them as a review source is fair to both sides of the gaming community. Please reconsider them as a source unless they are willing to give the same scores for multiplats on both sides, low and high, and concede that their current reviews for ps3 games are 10 or more points lower than even the lowest scores available. How can any site that is consistently lower that the average be taken seriously in this review driven market, unless metacritic is somehow a party to it? Even Variety's LBP review was completely based on conjecture, and not the merits of the games potential. Not to mention 15 points lower than the meta score and 10 points lower than the lowest score. Did you read that review Marc? The fact that in the past you would interpret the review and assign your own score is out of bounds. If a game reviewer cannot assign a score for all to see, it should not be open to interpretation or accepted. There is so much wrong with this site, the way reviews are half handed, weighted, and now with the mob user score and Marc your own admissions of interpreting voter intent make this site a mockery on an epic scale. How would I review your site, Marc?

This message has been edited. Last edited by: Gamer20,

Totally agree with Gamer20.

If you think sources like Variety are worthwhile, include them, but don't guess or communicate under the table for a score. If Variety doesn't have the balls to post a score, or they choose not to post scores with their reviews, Meta Critic shouldn't hard sell them on giving a score.

quote:

Originally posted by Crispy Taco:
Totally agree with Gamer20.

If you think sources like Variety are worthwhile, include them, but don't guess or communicate under the table for a score. If Variety doesn't have the balls to post a score, or they choose not to post scores with their reviews, Meta Critic shouldn't hard sell them on giving a score.

I doubt their reasoning for not posting a score is a lack of balls. Variety obviously doesn't use a scoring system.

I don't understand why you would concern yourself over this. The scoring of opinion is pretty shady in itself; you're never going to get a 100% accurate figure, and how do you know that each individual review for a game has been reviewed via a valid and unbiased format?

I checked out the Variety web site and reviews and it seems like a good site with some interesting review writing, however there does appear to be a bias towards the xbox 360. For eg with the LBP review..."requires a $400 Playstation 3 and $60 game" as though this were a downside to aquiring LBP. What has that got to do with reviewing the actual game? Hmmmm critisising the price of the PS3 is fanboy territory and a bit last year.

quote:

Originally posted by Lykon:
I checked out the Variety web site and reviews and it seems like a good site with some interesting review writing, however there does appear to be a bias towards the xbox 360. For eg with the LBP review..."requires a $400 Playstation 3 and $60 game" as though this were a downside to aquiring LBP. What has that got to do with reviewing the actual game? Hmmmm critisising the price of the PS3 is fanboy territory and a bit last year.

Although I agree that the PS3 is overpriced, I also agree that it's price shouldn't be used to lower the score of a game.

Regardless, I wouldn't be suprised if there was some bias in review magazines/websites, the same way Playstation Official Magazine UK, Playstation Universe, Team Xbox, or NintendoWorldReport may be biased.

Having just read the Variety review for resistance two I am now convinced that this is a website aimed at appeasing xbox360 owners. Most of the review is focused on comparing the xbox with the playstation. There are currently more xbox360 owners so it could be a deliberate buisiness strategy as well as personal bias. Your bound to get this type of thing going on in a competative industry based on subjective opinions.

I love metacritic and am very happy with this site it is very useful, thank you and well done Marc.

variety website has proven that they are biased,,,,let's not focus on exclusives for now.


They keep submitting low score multiplats for PS3 and high score multiplats for 360.

this could be due to a check submitted by Microsoft to this website so that they can keep saying that their library has the most "AAA" titles,while the truth is that metacritics is manipulating the data by accepting some reviews and not all(especially the ones that are biased against PS3)

for instance metacritics added a score of 100/100 for fable 2 from a website called Level which is a blog in Czech language(without any link and reference to the website).I don't see why this site was added other than sole purpose of pushing the meta score of fable 2 over 90.

or another example is that they put 100 for A out of A+ from 1up for a 360 exclusive game(Gears of war 2).

This all will not take a genius to figure that this very site is trying to manipulate the meta score .

websites like N4g.com accepts all the reviews as long as they are not from a blog.that way we know that the website is not manipulating the scores by choosing only biased sources for one site.

I really sense that the person in charge of submitting the review scores for this site is heavily biased /or paid a big check by Microsoft( cause MS is known for these kinda dirty practices in businesses).

why ps3 fans are going nuts?cause their numbers are increasing and they can not take this kinda obvious fraud and bias from a so called multiplatform website.

it's because of this..

_____________________________________


http://www.microsoft.com/presspass/press/2007/Sep07/09-19MSNVarietyPR.mspx



MSN and Variety Forge Distribution Relationship

MSN continues to add great content from leaders in entertainment.
REDMOND, Wash. — Sept. 19, 2007 — MSN® is joining forces with Variety.com to bring even more entertainment information to the site’s visitors. The distribution relationship will provide Variety with an avenue for reaching an even wider global audience and at the same time offer MSN a broader range of information on television, movies, music and Internet deals.

“For decades, Variety has set the standard in reporting on the entertainment industry,” said Jeff Dossett, executive producer and general manager for MSN. “We’ve seen a growing appetite on MSN for Hollywood information, and this collaboration will help us continue to offer our audience the best source for what’s happening in the entertainment industry.”

“Variety has long provided the best source of information to meet the needs of entertainment industry professionals. Variety.com also allows us to provide the kind of insider insights and scoops that the broader entertainment enthusiast wants,” said Charlie Koones, president and publisher of Variety. “Our distribution partnership with MSN will allow their users to enjoy the kind of in-depth entertainment coverage only Variety can provide.”

The Variety content on MSN covers a broad range of entertainment topics, including breaking news, in-depth coverage of award shows and film festivals, film and television, reviews, and video coverage of daily entertainment news and celebrity interviews. Over time, additional columns and blogs will be added including weekly columns from Editor in Chief Peter Bart. The most popular blog within the film community, Anne Thompson’s “Thompson on Hollywood,” which covers the film industry, and its exclusives, will be added, as will Cynthia Littleton’s “On the Air,” which offers a frank perspective on all things television. Beginning today, MSN users will see top entertainment stories from Variety on MSN Entertainment.

About MSN and Windows Live

MSN attracts more than 465 million unique users worldwide per month. With localized versions available globally in 42 markets and 21 languages, MSN is a world leader in delivering compelling programmed content experiences to consumers and online advertising opportunities to businesses worldwide. Windows Live™, a comprehensive set of personal Internet services and software, is designed to bring together in one place all the relationships, information and interests people care about most, with enhanced safety and security features across their PC, devices and the Web. MSN and Windows Live will be offered alongside each other as complementary services. Some Windows Live services entered an early beta phase on Nov. 1, 2005; these and future beta updates can be found at http://ideas.live.com. Windows Live is available at http://www.live.com. MSN is located on the Web at http://www.msn.com.

About The Variety Group/Reed Business Information (RBI)

The Variety Group — Daily Variety, Daily Variety Gotham, Weekly Variety, Variety China, Variety.com, 411 Publishing, Video Business and Tradeshow Week — are all owned by Reed Business Information (RBI), the largest business publisher in the U.S. RBI is a member of the Reed Elsevier Group plc (NYSE: RUK and ENL) — a world-leading publisher and information provider operating in the science and medical, legal, education and business-to-business industry sectors.

quote:

Originally posted by Spartan2008:
variety website has proven that they are biased,,,,let's not focus on exclusives for now.


They keep submitting low score multiplats for PS3 and high score multiplats for 360.

this could be due to a check submitted by Microsoft to this website so that they can keep saying that their library has the most "AAA" titles,while the truth is that metacritics is manipulating the data by accepting some reviews and not all(especially the ones that are biased against PS3)

for instance metacritics added a score of 100/100 for fable 2 from a website called Level which is a blog in Czech language(without any link and reference to the website).I don't see why this site was added other than sole purpose of pushing the meta score of fable 2 over 90.

or another example is that they put 100 for A out of A+ from 1up for a 360 exclusive game(Gears of war 2).

This all will not take a genius to figure that this very site is trying to manipulate the meta score .

websites like N4g.com accepts all the reviews as long as they are not from a blog.that way we know that the website is not manipulating the scores by choosing only biased sources for one site.

I really sense that the person in charge of submitting the review scores for this site is heavily biased /or paid a big check by Microsoft( cause MS is known for these kinda dirty practices in businesses).

why ps3 fans are going nuts?cause their numbers are increasing and they can not take this kinda obvious fraud and bias from a so called multiplatform website.

there we go,,,,,,didn't i mention in the post above you that MS is known for dirty practices in the business?

thanks man for backingg my post with official documents.

This website is a pathetic fanboyish little joke...

You are better off trusting N4g.com who doesnt do "selective" approvals of reviews, doesnt publish score guesses (variety) , doesnt publish blog reviews and generally is free of bias.

This website is a joke...

quote:

Originally posted by LordRaoh:
it's because of this..

_____________________________________


http://www.microsoft.com/presspass/press/2007/Sep07/09-19MSNVarietyPR.mspx



MSN and Variety Forge Distribution Relationship

Well done, that explains everything, it's very interesting, Thanks.

As for the games editors rather patronising statement above;

"...So if you're disappointed with any publication's review of a particular game, step away from your conspiracy theories and know that I've always been the custodian of Metacritic Games and I take this process extremely seriously...."

Then remove Variety scores from the website if you really do take it seriously. The evidence is all there in black and white.

^^ i doubt how seriously he is taking his job when he publishes wrong scores (1up gave GeoW2 A/A+ and metacritic lists it as 10/10...) , when he is missing a ton of reviews from PS3 multiplats so that metacritic on 360 is higher and when he uses Variety, a totally "unbiased" source that is owned by Microsoft , has reviewed Fable 2 9.8/10 way above the general critic concessus of 89% and has reviewed every single PS3 exclusive way below the general critics concessus.Latest examples LBP which got an 8/10 (the lowest , when its averaging a 96% easily) and Resistance 2 with a 7/10...

He is a joke if he considers himself taking his job seriously... not only that , but at the same time insulting our intelligence with such comments and actions...

Man you guys are just full of epic fail conspiracy theories.

quote:

Man you guys are just full of epic fail conspiracy theories.

or you are just ignorant to the facts.

I know what kind of people the metacritic editor's are and I also have a firm belief in Occam's Razor.

quote:

I know what kind of people the metacritic editor's are and I also have a firm belief in Occam's Razor.

well, you've got all the bases covered then, don't you. the only thing is the razor tells me you are part of the problem.

since we already have proof of a deal between MS and Variety, and we can clearly see the wild "randomness" of the scores and systems in place here at MC with a dictatorship over that system, then the fewest amount of assumptions would lead us to believe that something is at work here other than chance, coincidence, bad luck, a common mistake, or any other long line of assumptions that you, my dear liberalkid, would have to make in order to back up your claim of epic fail conspiracy theories by the majority here in this thread.

why don't you try to explain the relationship between MS and Variety, the ultra low scores for ps3 exclusives from Variety, the Variety reviews missing from one platform but present for another, the miscalculation from letter grades such as A/A+ to 100/100 or B+ to 83, the fact that many outstanding reviews for resistance 2 that would put it over 90 aren't published, the games editor admitting "gleaning" the review score from the overall impression of the review (which is just inexcusable), the sending of review scores via email now that the "gleaning" issue was exposed instead of just publishing the score on the parent site, and probably much, much more. So far that's all i've learned in a weeks time, without trying, so i'm sure there is more.

So, please, address these issues or refute them. Do something other than providing a worthless opinion.

So how does Variety's 85 for Gears 2 factor into this? That's the second lowest rating out there for that game.

metacrapic are one of BILLS BOYS and M$ now!

not to worry my fellow ps3ers we all know shitbox360 has passed its peak and ps3 iscoming into its own! ign for proper scores!