I don't make many polls.

Vote on whether you think MC allows too many albums in their Best of Year lists which have less than 15 reviews.

Make sure you vote AND leave your name and opinion. If you think that 7 is too few, but 15 is too many, put what the number should be. When I get enough votes, names and minimum review numbers, I'll plead our case (if any) to the higher ups. I am planning on listing ALL members who support raising the minimum number.

Question:
Should MC raise the requirement for Top End-of-Year lists from 7 to 15?

Choices:
Yes
No

 

I'll vote no, conditionally. If some of the really heavy hitters (Pitchfork &c) are the ones that are reviewing, then I see no problem with only 7 people reviewing (although I think 10 or 11 is a better number). If it's someone like the Topeka Review of Music, then, yes, you need more people to vote. Good idea to make this poll, though.

I voted yes, for the simple reason that if Metacritic's goal as a site is to reflect critical consensus, users aren't getting an accurate snapshot of what critics collectively think are the best records of the year.

Typically, you end up with a couple of oddball albums at the top that were only reviewed by a few sources. Personally, I'm fine with the oddball stuff being there, but if I were running Metacritic, it would concern me that users aren't seeing a truly accurate picture of critical consensus.

I don't believe we use Topeka, so what is a solid number. This is serious shit.

I think 15 is a good number, but I'd probably vary it depending on how many publications are in the pool that MC draws reviews from.

I think 7 is ok as long as a few "big names" are reviewing the album.

I voted yes for a 15 review minimum, or at least a 10 review minimum, because it would definitely make the list more credible. Some good albums may well get F'd in the A, but that happens already at the other end of the spectrum when they get pushed out.

How about a minimum of 10 instead?

7 to 15 seems like quite a substanial leap, i feel a middle ground is more warranted. Make it a nice big round number.

I'm voting yes. Surely it should require a fairly good number of reviews to be considered a "best reviewed" album, which is essentially what Metacritic compiles with the metascores. 15 is not really all that many. In Rainbows has 40 for crying out loud.

My vote is a solid 'no.' I like that Metacritic acknowledges and ranks albums from 'smaller' artists that don't get the press and attention of their more well-known contemporaries.

Additionally, raising it to 15 would mean that two albums in my top five would be out. I'd probably boycott the final album list in general if that were the case.

I voted for yes; however I would be a bit harsher and say that the minimum be 20 reviews. I think that this is fair because the more reviews an album receives, the more convincing it seems. Even if “the big hitters” are the ones that are reviewing it, some of the others are worthy as well. All of the publications that MC chooses are solid and credible.

Either way, even if it gets upped, there will still be that Field album that gets maybe 16 reviews and has a 91 and that Arcade Fire album with 42 reviews that has an 89. If people truly checked each album in a detailed manner—which they don’t—they’d see the latter is clearly superior.

First of all, I think From Here We Go Sublime is a stronger record than Neon Bible.

I understand that maintaining a higher average from a more substantial number of reviews is a notable feat, but I don't think that these albums from obscure artists on obscure labels should suffer simply because their releases aren't reviewed by as many publications. They've all received over 10 reviews, which I think is a perfectly fair cut-off point, and each have included reviews from some 'notable' sources (though this, of course, depends on who you're asking), so for crying out loud, just keep 'em in.

quote:

How about a minimum of 10 instead?

7 to 15 seems like quite a substanial leap, i feel a middle ground is more warranted. Make it a nice big round number.

I agree with JP.

quote:

First of all, I think From Here We Go Sublime is a stronger record than Neon Bible.

hell yeah.

quote:

Originally posted by Platypus Quest:
First of all, I think From Here We Go Sublime is a stronger record than Neon Bible.

I understand that maintaining a higher average from a more substantial number of reviews is a notable feat, but I don't think that these albums from obscure artists on obscure labels should suffer simply because their releases aren't reviewed by as many publications.

+1

Long Answer:

quote:

Originally posted by EricG75:
... Personally, I'm fine with the oddball stuff being there, but if I were running Metacritic, it would concern me that users aren't seeing a truly accurate picture of critical consensus.

If Metacritic really listed all the available sources then I don't see how they could give anymore of an accurate or complete picture of the critical consensus.

Furthermore, I understand that metacritic already lists a weighted average score, giving more influence to certain publications. I actually think that has more of an influence on the score than the number of reviews.

For example:

And Their Refinement Of The Decline by Stars Of The Lid has 12 reviews which AVERAGE (mean) to a score of 81.5 and give a MEDIAN score of 84, yet it's metacritic score is 88. Presumably this is because "Sputnikmusics"'s out of place score of 30 is given a lot less weight (say only 5% of the total rather than an equal 8%) than "allmsuic"'s 90.

Meanwhile, Radiohead's In Rainbows - with a whopping 41 reviews - was given a metacritic score of 88, even though it has an AVERAGE of 83 and a MEDIAN of 80!

Now you may be thinking "this pixies punk needs to stop wasting our time with pointless numbers and his elementary school usage of excel". However, given that the point spread between even just the top album of the year (burial at 91) and number 30 (grinderman with 83) is only EIGHT!, every single 'point' in the score counts in determining whether a deserving album can squeeze in to that coveted best of page or get relegated to abandonment and hurt feelings.

But then you already knew that right, since we're in this discussion. What I'm trying to say is that if we are going for magic numbers on a whim here it should be taken into consideration that: deciding which publications are more reliable/important and giving a numerical value to a review based on it's content (when it is not given) or when that number is often random/superficial/irrelevant (Pitchfork) is highly subjective. For instance, I don't like Pitchfork or think that their reviews/scores should be given much of ANY input into the consensus score. Nevertheless, I have a feeling that they may be one of metacritics more heavily weighted elites.

Thus, while it's important to have enough reviews to be able to get an average, I think by 7, or 9, or 14, there can be just as much of a metacritically 'accurate' score given as there can with say, 41 reviews.

Also, having a higher requirement might unfairly exclude albums that aren't pop/rock and therefore get less reviews. This is simply because there are fewer publications covering that type of music. This might include albums in the electronica 'genre', which seem to be dominating the best of list this year. Maybe it would be more useful to have the lists by genre. However, that would open up a completely different Pandora's box, especially given MC members tenacious opinions about genrefication .

Short Answer: No.

I think a lot of the best albums are completely overlooked by mainstream review publications (like The Rolling Stone), yet should still be available as the best album of a certain year. My favorite album this year is Breathe Owl Breathe - "Canadian Shield" and that was not reviewed anywhere, so I think there is more to a great album than which reviewers think it is great.

Holy crap when I started writing my above post there were only like 5 responses. I stayed up till 11 doing that shit . Work better be canceled tomorrow. Ice baby, ice!

this my first post.. but i've been visiting the site for about 3 years... thought i'd weigh in here

A lot of sites weight albums based on the number of reviews something receives.. I dunno the equations they use, but it would def be a better idea instead of putting a definite limit on it.
I don't know if it's a good idea overall, though.

If an album only gets reviewed 6 times, but gets scores of 100 on all of them.. that's worth hearing about at the end of the year even if most of the major sources slept on it.

I think that if metacritic is already patrolling the sources of reviews and paring them down to a reasonable number, that adding a further 'paring down' shouldn't be necessary.

BUT it would be very interesting to see a 'populist' top albums of the year list as well as one without all the restrictions where, say, it only includes albums with 25.

just some ideas...

I voted yes.

Instead of re-inventing the wheel, here's a post I made on this subject about two weeks ago:

quote:

I'm very annoyed to see Burial at the top of the list. It just shows that the best rated albums list is meaningless.

Look at the top three albums. All are electronic albums that do not appeal to a wider base, so they receive reviews in only limited publications. Burial has only 10 reviews. Stars on the Lid has only 11. The Field has only 14.

Compare that with Radiohead - 88 (37 reviews), Arcade Fire - 87 (42 reviews), The National - 86 (31 reviews), Elliott Smith - 85 (30 reviews), Spoon - 84(33 reviews), and Iron & Wine - 84 (33 reviews).

I think it is much more impressive to have an 88 rating averaged from 37 reviews than a 92 rating from 10 reviews.

I think the current system of only 7 reviews skews the list towards electronic albums that tend only to get reviewed in more specialized publications.

I don't like the choices. I'm voting for Nader.