quote:

Originally posted by The Fall of Troy:
I voted yes.

Instead of re-inventing the wheel, here's a post I made on this subject about two weeks ago:

quote:

I'm very annoyed to see Burial at the top of the list. It just shows that the best rated albums list is meaningless.

Look at the top three albums. All are electronic albums that do not appeal to a wider base, so they receive reviews in only limited publications. Burial has only 10 reviews. Stars on the Lid has only 11. The Field has only 14.

Compare that with Radiohead - 88 (37 reviews), Arcade Fire - 87 (42 reviews), The National - 86 (31 reviews), Elliott Smith - 85 (30 reviews), Spoon - 84(33 reviews), and Iron & Wine - 84 (33 reviews).

I think it is much more impressive to have an 88 rating averaged from 37 reviews than a 92 rating from 10 reviews.

I think the current system of only 7 reviews skews the list towards electronic albums that tend only to get reviewed in more specialized publications.

looks like both Burial and Stars of the Lid have a dozen reviews each now, which if I were to pick a cut off point, that would be it, so not a lot would change on the list as of now.

one thing that could change things is if metacritic included the Users Score as equivilant to one review.

otherwise I would suggest that metacritic use some sort of formula to adjust up slightly those albums that get massive amounts of reviews and still manage to stay near the top of the list.

No. I repeat; NO!

I couldn't care less if an album has 2 reviews. I read Metacritic review compilations for the convenience of having many reviews in one place, not to be told that 31 critics thought something was good.
Point in fact, I find The National, Arcade Fire, Spoon and Iron & Wine albums bloody rubbish, so a large consensus is neither here nor there.

One single well written review is enough for me.
I find it nonsense to be shutting out amazing art from what is essentially an information resource (M/C) due to some arbitary number.

Seriously, who cares where an album sits in some fucking list?
Burial's Untrue is a far superior album to say, Spoon, but I couldn't give a fuck if it was bottom ranked.
Its enough to know that I know its excellent.

Voted no as well, reasons have all pretty much been covered by all the no-voters already. I haven't heard the Burial or The Field albums, I might like them or I might not... but like Ishmael said, why on Earth would you be bothered that they're sitting above albums you like on a list? I'm happy that they are just there, as a highly regarded recommendation - if the cutoff was higher and those albums didn't make the list, I might not even know they exist. Then I wouldn't be able to even check them out, and decide for myself how good they are.

P.S. Spoon bloody rubbish!?

I vote no.

I like things the way they are because it introduces me to cool new music I would never had tried otherwise. Both The Field and Burial I had seen before on Pitchfork.com, but I didn't want to get them because they only scored a 9.0 and 8.4 and the reviews made them sound like they weren't real music, but when I saw The Field as #1 here on Metacritic, I thought, "Hey, I should really give that a try." And I loved it! Still do. Burial is a bit of a different story, but I'm still glad I tried it out because "Archangel" is an amazing song that I probably would never have heard otherwise.

Besides, alot of great CDs that get alot of reviews still make it way up there on the list, like TV on the Radio last year, or the last two Outkast CDs.

In fact, I think instead of changing the number of necessary reviews, I think the lists should be extended instead. Like a top 50 for the year (So that the Arctic Monkeys could still be on the list ) and a top 300 of all-time. I don't get as upset about seeing albums I like getting knocked down as much as I do about seeing albums that I love being knocked off.

I would like to add an ultimatum of sorts.

If, in the event of the "yes" votes wining. Mark I'd like you to add me to the list of the people who support when you present the case to Metacritic.

However, 15 is too many, maybe 10, 12, or 14. The highest I think it should go is double the current. 15 is just way too many.

I still vote no. But I also feel like I could almost go either way.

I voted yes, but, like STS, really see both sides of the issue. Although I really like the meta score (more on movies and DVDs than music) as a rapid measure of whether I should consume the item in question, the fact is that the music score generally only points me towards the albums I'd like to know more about.

Example, the Burial metascore encourages me to check out the four or five publications whose tastes I trust, and read their actual reviews. A quick perusal tells me that I'm not really likely to like the album, despite the acclaim in which it's held. It's also true that this poll addresses the place in the year end list only. The metascore would remain the same. Burial would still have a 91, it just wouldn't end up at the top of the list.

On the other hand, I think the number of publications giving high marks to a record does tell you something about a given work's ability to penetrate the media. Radiohead is, predictably, going to make a splash across many outlets. But I think it means something that a smaller, "genre specific" act like Battles, was reviewed by 31 outlets. Doesn't it say something about the zeitgeist, that Battles or LCD Soundsystem are reviewed in as many publications as they are? Does it say something, again, that their high scores are sustained against a large range and number of reviewers rather than simply in "specialty" journals. I think that more reviews might be reasonable, again, maybe 10 or 12 rather than 15.

Finally, I wonder if some additional mathematical adjustment to the formula, reflecting the number of the reviews would be valuable. 15 reviews gets an adjustment factor of 1. 10 to 14 gets your score multiplied by 0.95, 5-10 reviews gets your metascore multiplied by .92, etc. On the other end, 15 to 20 reviews gets you a "fudge factor" of 1.05, etc.

quote:

Originally posted by EricG75:
I voted yes, for the simple reason that if Metacritic's goal as a site is to reflect critical consensus, users aren't getting an accurate snapshot of what critics collectively think are the best records of the year.

Typically, you end up with a couple of oddball albums at the top that were only reviewed by a few sources. Personally, I'm fine with the oddball stuff being there, but if I were running Metacritic, it would concern me that users aren't seeing a truly accurate picture of critical consensus.

i think that i agree with you but i also agree with

quote:

How about a minimum of 10 instead?

7 to 15 seems like quite a substanial leap, i feel a middle ground is more warranted. Make it a nice big round number.

voted yes...

I voted yes.

Nothing wrong with Metacritic collecting the reviews for albums with only a few reviews, and giving a number, but the playing field is not level when one album gets 50 reviews and another gets 7.

I don't want to exclude smaller albums that get limited reviews but are still excellent, just because many publications are only interested in the big, well-known albums, but I think there are enough print and online publications now that most albums get at least 15 reviews. Maybe 15 is too many, but 7 is too few.

quote:

Posted by thefanste
P.S. Spoon bloody rubbish!?

You know I'm right...

Perhaps they should have a "Special Mention" section for albums that get high scores but have less than 15 reviews. This subcatagory would give people something else to look at on the site. Much in the same way that I check P4k's "Recommended" section almost as much as their "Best New Music" section, I'm sure people would regularly check the "Special Mention" section to find underappreciated gems.

quote:

Originally posted by Ishmaels Coffin:
No. I repeat; NO!

I couldn't care less if an album has 2 reviews. I read Metacritic review compilations for the convenience of having many reviews in one place, not to be told that 31 critics thought something was good.
Point in fact, I find The National, Arcade Fire, Spoon and Iron & Wine albums bloody rubbish, so a large consensus is neither here nor there.

One single well written review is enough for me.
I find it nonsense to be shutting out amazing art from what is essentially an information resource (M/C) due to some arbitary number.

Seriously, who cares where an album sits in some fucking list?
Burial's Untrue is a far superior album to say, Spoon, but I couldn't give a fuck if it was bottom ranked.
Its enough to know that I know its excellent.

By this logic you should be for lowering the minimum review cutoff to 4 or 5 reviews, or less. It's arbitrary as well.

I'm not positive, but it seems like there might be some confusion as to what Mark is actually proposing.

Albums with less than 15 reviews would still show up on the site (I believe anything with 5 reviews does), they just wouldn't show up in the "Best of 20XX" section. That section already has additional criteria such as no reissues or greatest hits albums.

So, no one's getting short changed out of any information, the only thing that would change is that the "Best of 20XX" would more accurately reflect critical consensus.

I'm suspecting this whole discussion arose because some folks are mad because some lesser known (though quite wonderful) electronic records are beating their beloved Radiohead for the top spot in 2007. Big effing deal.

The suggestion at hand, however, IS a big effing deal. Skewing the system to require more reviews is unfair to albums that don't get the press they deserve. I understand that these highly-ranked, under-reviewed albums would still maintain the same metascores, but excluding them from the Best Of lists is just not cool, considering that the albums that would be shut-out are pretty damn great -- as illustrated by their inclusion in many of our own Best Of lists here on the forums.

I understand the accomplishment of averaging a high score over a large number of reviews -- really I do -- but less reviews doesn't necessarily correlate to a "lesser" album.

Thus, the "Special Mention" section.

quote:

Originally posted by Platypus Quest:
I'm suspecting this whole discussion arose because some folks are mad because some lesser known (though quite wonderful) electronic records are beating their beloved Radiohead for the top spot in 2007. Big effing deal.

The suggestion at hand, however, IS a big effing deal. Skewing the system to require more reviews is unfair to albums that don't get the press they deserve. I understand that these highly-ranked, under-reviewed albums would still maintain the same metascores, but excluding them from the Best Of lists is just not cool, considering that the albums that would be shut-out are pretty damn great -- as illustrated by their inclusion in many of our own Best Of lists here on the forums.

I understand the accomplishment of averaging a high score over a large number of reviews -- really I do -- but less reviews doesn't necessarily correlate to a "lesser" album.

I'm sorry, I don't get this argument. Nobody is saying that lesser known albums should not get recognition on Metacritic. Lesser known albums do get recognition on Metacritic. In fact, I would categorize about 95% of what goes on people's year-end lists as "lesser known albums." Of the 50 or so albums that I purchased this year, I think I would probably have heard of only 3 of them (Icky Thump, Sky Blue Sky, and In Rainbows)if it were not for Metacritic giving them recognition.

Albums with 4 reviews from the publications that Metacritic lists do get recognition on Metacritic. In fact, all recent releases that are on Metacritic get placed in a column on the left hand side of the screen. You can still see Burial's 92 over there and link to it to read the reviews.

But, that's not what the best-of lists are for. They are for recognizing those albums for which the critical consensus resulted in the highest score. And, no matter how you look at the issue, both sides have to admit that the playing field is not equal.

In essence, what those voting against the proposal is that they want to give lesser known artists the advantage over artists like The National, Les Savy Fav, Iron & Wine, Okkervil River, Panda Bear, etc., who, despite being unknown to the average music fan, have earned wide-spread critical attention based on the quality of their work.

I'd say ten or twelve is a good cutoff point. Agreeing with what Ishmael said, that seeing an album that high on a list piques my interest, not necessarily convinces me that it's a better album than something lower down. You have to wait for the year-end lists to get a true critical consensus on the 'best of the year'.

I voted yes; and I think the minimum amount of reviews for an album should be 15-20 at least to be ranked in overall list, since the only critics who review lesser known albums tend to have a bias towards certain genres, artists.

quote:

Originally posted by efrigo:
I'd say ten or twelve is a good cutoff point. Agreeing with what Ishmael said, that seeing an album that high on a list piques my interest, not necessarily convinces me that it's a better album than something lower down. You have to wait for the year-end lists to get a true critical consensus on the 'best of the year'.

I agree with this. After all, even the scores themselves do not necessarily predict where the critics are going to put them at the end of the year. These less popular works should remain at the top. If you decide to remove them, you are depriving potentially interested audiences of knowledge of their existence.

It wouldn't be "depriving" any audiences. What makes 7 the magic number? Aren't they depriving audiences of great albums with 6 reviews? It's just a nearly arbitrary number that was bound to need future revision. Are you nuts?

quote:

Originally posted by goathouse:
It wouldn't be "depriving" any audiences. What makes 7 the magic number? Aren't they depriving audiences of great albums with 6 reviews? It's just a nearly arbitrary number that was bound to need future revision. Are you nuts?

It's as arbitrary as any other number you'd come up with. There are a number of decisions made in the compiling of that list that are rather arbitrary -- the weighing of each critic, the very selection of critics (Film Threat over FFC or Slant? Really?) -- as well as things outside the list that still affect something's placement (A 93 instead of 94? How precise of you!)

That's why I think it's best to let the critics' end-of-year lists speak for themselves to determine the overall recipient of the most accolades and allow the very flawed list to occasionally send us to some lesser-known artists who remain outside the mainstream tunnel vision.