Hmm. There doesn't seem to be a "discuss Metacritic" forum. I wonder why?

I was thinking that it would be useful to have a system whereby the reviewer is weighted, and that would affect the final metascore an item reviewed.

Clearly Metacritic themselves cannot weight the value of reviewers, but the rest of us can.

There are two possible ways of accomplishing this, and I am suggesting that both be employed in addition to the existing system.

The first is personal reviewer reviews. For example, in PC Games for instance, I might think that Stuff Magazine's opinion on a shootemup is worthless. So I would give them .1 value. If I thought PC Gamers opinion was the best, I would give them 1. So rather than the metascore being the average (sum(scores)/ reviewers), and all reviewers equal, my personal metascore would be based on the reviewers I think have greater value.

The second way is simply the meta version of the first, where everyone votes on the value reviewers and they accumulate an average metascore which would provide their weighting.

I suppose we should take that to its logical absurd extreme and vote for each other so that we can decide how much someone elses vote counts.

koan

quote:

---

Originally posted by koan:

Clearly Metacritic themselves cannot weight the value of reviewers, but the rest of us can.

---

Why is it that you think we (metacritic) cannot weight the value of reviewers? I think we are at least as qualified as anyone else. We've read hundreds of thousands of reviews from hundreds of games pubications over the last 10 years. Furthermore, we've studied the industry very closely over the last 5 years, carefully considering the relative merits of its critical publications. We don't disclose the weights we assign to each publication, but it's a safe bet that we hold PC Gamer in a similar high regard as you do, and we would likely see eye-to-eye on the frivolousness & satirical nature of Stuff's reviews. Your suggestions may or may not be easy to implement, but we'll certainly consider them. However, I'd be curious to know why you think that "Metacritic themselves cannot weight the value of the reviewers, but the rest of (you) can."

Thanks...

Does this site get enough traffic to make a reader opinion-centric view of the critics a realistic option?

quote:

---

Originally posted by Member 27:
Does this site get enough traffic to make a reader opinion-centric view of the critics a realistic option?

---

In the film section, we invite our users to give feedback & "vote" for or against the various critics, but that input doesn't factor into the metascore formula. We don't have a similar section in games because most publications don't have a set group of critics that always review new games.

We also obviously have a user voting area for all films, albums, and games. What we don't offer, at this point, is the ability for users to add or delete critics from our slates in the given genre, or to apply their own weights to those critics. We may eventually get to that level of customization, but for now, you'll have to trust us.

quote:

---

Originally posted by Metacritic Games Editor:

quote:

---

Originally posted by koan:

Clearly Metacritic themselves cannot weight the value of reviewers, but the rest of us can.

---

Why is it that you think we (metacritic) cannot weight the value of reviewers? I think we are at least as qualified as anyone else. We've read hundreds of thousands of reviews from hundreds of games pubications over the last 10 years. Furthermore, we've studied the industry very closely over the last 5 years, carefully considering the relative merits of its critical publications. We don't disclose the weights we assign to each publication, but it's a safe bet that we hold PC Gamer in a similar high regard as you do, and we would likely see eye-to-eye on the frivolousness & satirical nature of Stuff's reviews. Your suggestions may or may not be easy to implement, but we'll certainly consider them. However, I'd be curious to know why you think that "Metacritic themselves cannot weight the value of the reviewers, but the rest of (you) can."

Thanks...

---

Heh, my mistake, I wasn't aware that Metacritic already weighted the reviewers.

The reason why I presumed that you didn't wasn't because you weren't able or competant, more that I throught that this would expose Metacritic to all kinds of legal wrangling.

I assumed that you would treat reviewers equally so as not to get their lawyers calling. Impartiallity.

Now I find that you do rate the reviewers, but you don't disclose those weightings. That is interesting in itself. Presumably you don't disclose them for the same reason I didn't think you weighted them in the first place?

koan

No, we don't disclose the specific weights that we apply to each publication/critics because that's our secret formula, our work product, something that we've spent a great deal of time coming up with through studying the industries, talking to readers of the respective publications, etc. There's absolutely no "legal" reasons why we choose not to disclose that stuff. And I certainly wasn't interpreting your earlier comment as suggesting that we're incompetent (although you're free to think so!!).

Marc

I personally dont really care how they are calculating the scores because they seem to be very accurate and well in line with my own opinions on things. This site is a great indicator of whats good and whats not.

I think the point is that metacritic is hoping to meet the expectations of everyone. While individual advocacy is an indicator that they are successful, it only extends to that person advocating them.

I find metacritic a useful starting point, but always find myself scanning the list of reviewers to see who contributed the scores that gave the >80-90. If I could weight the reviewers myself, than that step would be unnecessary.

Game Editor values the "behind the scenes" process of Metacritic, whereas I believe that people are able to judge for themselves, yet still value the collating process that metacritic offers. It is the service and branding that allows metacritic to retain its position. At this stage, I would say their brand equity is the strongest thing they have.

But lets not see it beyond that. A unique business model works for the duration that it is unique, and it only remains so for the length of time that it takes for someone to duplicate the idea with enough twist that it attracts attention.

The way to step ahead of the game is to provide the service no one else has thought of, which at metacritics inception was metacritic itself. It is time to extend.

koan

quote:

---

Originally posted by koan:

I find metacritic a useful starting point, but always find myself scanning the list of reviewers to see who contributed the scores that gave the >80-90. If I could weight the reviewers myself, than that step would be unnecessary.

---

I don't know anyone who simply looks at the overall score and doesn't check out what at least some of the individual critics had to say. Even if you could weight your own critics, you'd be stupid not to check out some of the individual scores and quotes from the critics. More than just a linking service, the fact that this site provides "estimated" scores for such sources as the NY Times, LA Times, and others which don't provide scores is incredibly time-saving. Do you trust that they're getting it right? That's the key question.

quote:

---

Originally posted by Josie P.:
Do you trust that they're getting it right? That's the key question.

---

It is, and they are I think in the whole. My own weightings would confirm that.

Initially I was suggesting making an enhancement (at least from my perspective), simply because I thought it would be valuable to me.

Now I find myself thinking about this from a context of metacritics requirements to extend the service it offers to continually stay ahead in the game (much like Google do).

I am sure that there are plenty of people at metacritic who do this anyway, so it is less important coming from me.

The most important thing for me right now is to see if I can retain my "Slacker" class on these forums irrespective of how many posts I have made

koan

quote:

---

st important thing for me right now is to see if I can retain my "Slacker" class on these forums irrespective of how many posts I have made

---

That can be arranged for a small fee ...