too heavily weighted ratings ? its outrageous - Progressive Rock Music Forum

Author

Message

Topic Search

Topic Options

Failcore

Forum Senior Member

Joined: October 27 2006
Status: Offline
Points: 4625

Posted: October 10 2008 at 10:07

I agree with both sides kind of. I think that being a collab should give you a little extra clout, but I think the main weight should come from how many reviews a person has done. As its mentioned, there is generally a direct proportionality between experience and review quality. Therefore, an ordinary joe who's written 1000 reviews should have more say than a collab that's written 50. Just my opinion. Discard it as you see fit.

Vibrationbaby View Drop Down

Forum Senior Member

Joined: February 13 2004
Status: Offline
Points: 6898

Posted: October 09 2008 at 23:46

I just gave a two star rating so there.

debrewguy

Special Collaborator

Honorary Collaborator

Joined: April 30 2007
Location: Canada
Status: Offline
Points: 3596

Posted: October 08 2008 at 19:43

I weigh 285 lbs. Can I get a more heavy weight attached to my ratings ? This would seem to be a very good way to objectively measure things. In return I promise not togive all Ange albums Fives, nor to rate all Krautrock albums a one. And if the T is nice to me I might see to it that DT gets more praise.
Or, if I must limited to a genre, I could specialize in heavy prog.

"Here I am talking to some of the smartest people in the world and I didn't even notice,” Lieutenant Columbo, episode The Bye-Bye Sky-High I.Q. Murder Case.

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 08 2008 at 15:10

russellk wrote:

There are two pressures on SC and prog reviewers, it seems. Some people (Yorkie X, for example) don't want to see many one-star reviews, while others (Finnforest, for example) lament the over-use of the five-star review. While both camps have compelling arguments, in effect this leaves us with three stars to rank hundreds of albums. That's simply too crude, in my opinion. Personally, I use all five stars in order to sufficiently distinguish between albums.

I've only regretted a single one-star review I've written - that for CYNIC's 'Focus', which was completely ruined for me by the over-processed vocals. (I don't mind growling, I love prog-metal, but these vocals were too synthetic for me.) I should have talked more about the innovative jazzy music, but I couldn't get past the vocals. The rest of my 14 one-star reviews (of 376 total - or 3.7%) were, I still think, justified (in my opinion).

My problem is in awarding too many five-star albums (63, or 16.8%). I genuinely think I've heard a hundred five-star prog albums out there - I'll get to the others later - and probably at least that many again (that with my tastes and experience I'd call five-star) I haven't heard.

I wonder how many one-star albums are out there? Hundreds, I expect. Trust me, at thirty New Zealand bucks a pop, I'm not going hunting for them!

I try to keep the balance, but there are too many good albums.

62	5 stars	25.73	%
103	4 stars	42.74	%
48	3 stars	19.92	%
18	2 stars	7.47	%
10	1 star	4.15	%
241		100.00	%

This gives an average of 3.78 stars, and the biggest incidence is in 4 stars.

Iván

russellk

Prog Reviewer

Joined: February 28 2005
Location: New Zealand
Status: Offline
Points: 782

Posted: October 08 2008 at 14:49

There are two pressures on SC and prog reviewers, it seems. Some people (Yorkie X, for example) don't want to see many one-star reviews, while others (Finnforest, for example) lament the over-use of the five-star review. While both camps have compelling arguments, in effect this leaves us with three stars to rank hundreds of albums. That's simply too crude, in my opinion. Personally, I use all five stars in order to sufficiently distinguish between albums.

I've only regretted a single one-star review I've written - that for CYNIC's 'Focus', which was completely ruined for me by the over-processed vocals. (I don't mind growling, I love prog-metal, but these vocals were too synthetic for me.) I should have talked more about the innovative jazzy music, but I couldn't get past the vocals. The rest of my 14 one-star reviews (of 376 total - or 3.7%) were, I still think, justified (in my opinion).

My problem is in awarding too many five-star albums (63, or 16.8%). I genuinely think I've heard a hundred five-star prog albums out there - I'll get to the others later - and probably at least that many again (that with my tastes and experience I'd call five-star) I haven't heard.

I wonder how many one-star albums are out there? Hundreds, I expect. Trust me, at thirty New Zealand bucks a pop, I'm not going hunting for them!

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 08 2008 at 12:04

Dean wrote:

No, I would NOT say that over use of the 1-star rating by a collab is abusive. It could be, but that does not mean that it will be. If you reviewed and gave 1-star to all the albums in Symphonic that currently have average ratings less than 2 then that would perhaps over-zealous, but it would not be abusive. If you do the same thing to all the albums in Crossover that have an average greater than 4, then that could be seen as abusive, vindictive and lots of other words ending in -ive - but mathematics and statistics cannot show why you would have done that.

Then you haven't read this thread, there's people who even saidthe actual use of 1 star ratings by collaborators is excesive:

Yorkie X wrote:

^^ That's too Many one star reviews if you ask me ... but since nobody asked me I`ll shut up Stern%20Smile

This was posted exactly after I posted this same list for the first time, so there is people who believe the use of 1 stars is excesive and abusive.

Yes, I do believe you decided to review those albums - as Peter said, to review them (or not to) is your choice, not your obligation - not even Hughes can review everything he hears.

I was asked by M@X and the teams were asked by the Adm Team (Before you were an Adm), to try not to leave albums without reviews, so some of us rated albums that we never would had cared about, and we did it with the most possible fairness, so yes some albums were chosen for us by the circumstances, nobody had reviewed them, we started to review those albums that otherwise we wouldn't had even cared about, and we did it to help the sirte after a request.

BTW: Even in the fantastic case of a bitter Collaborator who would rate all albums that have 2 stars (Something that doesn't happen in reality, because people rate essentially the albums they like) giving 1 star to albums that deserve 2 stars is abusive and having a very high percentage of 1 star ratings, could be considered abusive by some members,

But we will never agree, better agree to disagree.

Iván

Edited by Ivan_Melgar_M - October 08 2008 at 12:28

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 08 2008 at 03:15

Ivan_Melgar_M wrote:

Dean wrote:

2.46% shows that the average number of 1-star reviews by an SC is a low number - it does not disprove

That's really bad quote editing Iván

Ivan_Melgar_M wrote:

Dean, there are two forms of abusive usage:

Deliberate use of 1 star ratings in some albums to modify the charts: This doesn't come from Collaborators in years of experience here, bvut it can be verified also, only that would bneed a math expert and I'm not one
Excesive use of 1 star ratings: Tell me something....If the average of 1 star ratingsinstead of 2.46% was 24.6% in the Collaborators, Wouldn't you say that's abusive? I'm sure you would, but with this numbers you can verify that the SC's are making a very conservative use of 1 star ratings

So yes, I stand in my point, you can measure with numbers if the use of 1 stars is excesive and abusive.

Iván

Ivan_Melgar_M wrote:

BTW: I don't know if añl the Collaborators do this, but in my case I don't select bands, when I check a band or a new album, I try to review it, even if I don't like it.....Or do you believe I decided to review Opeth, Symphony X and Torman Maxt?

Yes, I do believe you decided to review those albums - as Peter said, to review them (or not to) is your choice, not your obligation - not even Hughes can review everything he hears.

What?

Vibrationbaby View Drop Down

Forum Senior Member

Joined: February 13 2004
Status: Offline
Points: 6898

Posted: October 08 2008 at 01:43

Well isn't that what I said ? I'd rather read a review by a fan rather than one of these monkeys. Can't understand why they get paid. As for Celine, she has nothing to do with music whatsoever.

Peter

Special Collaborator

Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669

Posted: October 08 2008 at 00:18

Well, goodnight -- that was almost fun, writing to myself like that!

I shoulda read a book....

"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.

Peter

Special Collaborator

Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669

Posted: October 07 2008 at 23:58

^ See me, Vibe, I don't read music mags I don't like....

Much like I wouldn't stay long in a place I hate....

In any case, I'm not defending or praising "professional," paid reviewers. (Nor do I think you automatically become a know-nothing-of-your-subject "moron" as soon as you start to get paid. On the contrary, professional reviews have led me to much great music over the years-- though admittedly, not much prog of late. Then again, as you and I know, all of the good, real prog was made over thirty years ago.....

)

No, I'm merely pointing out the fact that we are prog fans, and this inescapable fact is reflected in the numerous glowing reviews we write -- we overwhelmingly like the music we seek out and buy, as a matter of course. You don't buy your music randomly, do you, as a magazine reviewer is randomly sent albums?

There's a huge difference in fans who seek out and pay for the music, vs those who are paid to listen. Why is this obvious fact so hard for people to grasp?

Shall we send you a Queen Celine deluxe boxed set? Concert tix?

Edited by Peter - October 08 2008 at 00:15

"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 07 2008 at 23:57

Dean wrote:

2.46% shows that the average number of 1-star reviews by an SC is a low number - it does not disprove

Dean, there are two forms of abusive usage:

Deliberate use of 1 star ratings in some albums to modify the charts: This doesn't come from Collaborators in years of experience here, bvut it can be verified also, only that would bneed a math expert and I'm not one
Excesive use of 1 star ratings: Tell me something....If the average of 1 star ratingsinstead of 2.46% was 24.6% in the Collaborators, Wouldn't you say that's abusive? I'm sure you would, but with this numbers you can verify that the SC's are making a very conservative use of 1 star ratings

So yes, I stand in my point, you can measure with numbers if the use of 1 stars is excesive and abusive.

Iván

Edited by Ivan_Melgar_M - October 08 2008 at 00:00

Vibrationbaby View Drop Down

Forum Senior Member

Joined: February 13 2004
Status: Offline
Points: 6898

Posted: October 07 2008 at 23:35

I forget which thread I wrote this in but I think these professional reviewers for rags like Rolling Stone, The Wire, NME and the like are a bunch of sub-morons who know nothing about music and for the most part think artists like Beck, The Clash or Stereolab are musical geniuses. I would much prefer to read a review for a prog album by someone who at least has a passion, appreciation or knowledge of the genre. I must admit I've read quite a few great reviews here. Speaking for myself, I don't really have that much time to write reviews and the albums that I review actually come from my collection and I've slammed the ones that I believe deserve to be slammed. Others, even though I might personally be ape***t over them I'll review with the guidelines of the site in mind.

Henry Plainview View Drop Down

Forum Senior Member

Joined: May 26 2008
Location: Declined
Status: Offline
Points: 16715

Posted: October 07 2008 at 23:23

I'm more bothered by how far you have to go out of your way to even see non-collab reviews than by their weighting. The collabs should just be on top, not in their own box one must navigate away from.

if you own a sodastream i hate you

Peter

Special Collaborator

Honorary Collaborator

Joined: January 31 2004
Location: Canada
Status: Offline
Points: 9669

Posted: October 07 2008 at 23:13

Re the relatively low number of one-star ratings by PA Collabs (as my friend Ivan highlighted) , again (and as with the obverse -- the relatively high number of 4-5 star ratings), I think it must be remembered that as we are not professional reviewers, we are not given random albums to review.

We are overwhelmingly reviewing the albums in our own collections, which we bought -- generally with a fair idea of what we were getting into. We seek out stuff we can expect to like, from artists we have heard before, & liked. We like most of the stuff in our collections, don't we?

For good or ill, this is simply the inescapable nature of an unpaid, voluntary, fan-driven review site. it's not a question of all of us trying to be "fair", or to avoid being too negative -- we simply don't own much music to be negative about in the first place. It stands to reason, doesn't it?

However, with a magazine reviewer it's fundamentally different -- he is sent random stuff by the record labels, some of which he will quite like, much more of which he will considerer average, and plenty of which he won't like. But he is paid to listen (by the mag -- hopefully not the label) and to review -- good, bad or indifferent. It's his job -- not a labour of love hobby.

I don't think that "prog" (whatever the heck that is -- remember, it's NOT a single genre) therefore somehow contains less subjectively "bad' music than other forms of modern music. I'm sure that if I owned all of the albums, from all of the many genres listed here, I would -- proportionately speaking -- find plenty to dislike. If I reviewed a broad cross-section of all that is here, my percentage of one (and two) star reviews would, perforce, greatly increase.

I didn't like the one DT album I was given -- I gave it an unfavourable (but honest) review. As I am neither(as far as I can tell

) nuts nor a wealthy masochist, I did not then buy any further DT albums -- thus you saw no more (unfavourable) DT reviews from me.

I'll never like growling, or accept it as singing. To each his own, but it really irritates me -- thus I buy no albums which feature that vocal style, thus you'll get no (scathing) reviews of such albums from me. (Other than for the one Haggard album I was given, a few years ago.) Ditto most prog metal: I don't like it, I don't buy it, therefore I don't -- and can't --review it.

Surely most of us are similar: With limited funds, we don't purchase music randomly, let alone spend money for music we know or strongly suspect (from samples here, and also from more descriptive reviews) we won't like.

Thus, on a related note, I am very suspicious of the motives and methods of reviewers here who have panned several albums by a single artist. Are they buying multiple albums from artists whom they already know they don't like?

Are they borrowing "friends" collections of a given artist, merely to pan them?

Are they stealing (illegally downloading)

all of this music they don't like, just to torture themselves with the music, to write nasty reviews for no monetary recompense?

^ That's one petty, immature person!

Bottom line -- it's a fan site. FANS overwhelmingly write the reviews here. DUH!

There are some exceptions, I suppose ("Sean Trane," for one apparently borrows oodles of albums from the library), but for most of us, life's too short to listen to music we don't like for no real reward -- least of all to the degree and attention level required to review it.

(The afore-mentioned youngsters who write "revenge" reviews panning representatives of entire musical genres they despise, or albums which their enemies played at their sociopathic house-wrecking parties are another such exception... but those so-called "reviews" can usually be spotted by their sub-par, lack-wit, moronic English, their brevity, and overall lack of substance. )

Thankfully, they're often soon deleted -- thanks Bob!

Edited by Peter - October 08 2008 at 00:13

"And, has thou slain the Jabberwock?
Come to my arms, my beamish boy!
O frabjous day! Callooh! Callay!'
He chortled in his joy.

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 07 2008 at 20:45

Originally posted by <FONT color=#0000ff>Ivan_Melgar_M</FONT>

Ivan_Melgar_M wrote:

Dean wrote:

Then what did you mean by stating: "How many genres have 97.54% of good albums?"?

That for any determined gruup as the Spécial Collaborators, to find only 2.46% of one star ratings sounds very small to me.

...97.54% is produced by subtracting 2.46% from 100% (ie good albums = all albums - bad albums) - all I am saying is you cannot make that calculation since the result does not mean or imply a number of "good albums" in a genre, which is the question you asked.

Lets assume something, 1 star means a bad albuum FOR ANY REVIEWER, i doesn't mean it's intrinsecally bad album, but if a normal Special Collabotrators finds only 2.46% of bad albums, it's very small.

This doesn't mean the album is bad per se, only that from each 100 albums a normal SC rates, finds only 2.46 bad albums.

No - it means that the SCs only choose to review a small number of bad albums - nothing more, nothing less - it is not 100 random albums, it is 100 selected albums - so you cannot draw any conclusion other than that.

Originally posted by <FONT color=#0000ff>Ivan_Melgar_M

Ivan_Melgar_M wrote:

dean wrote:

2.46% shows that the average number of 1-star reviews by an SC is a low number - it does not disprove that the use of a 1-star rating by an SC is excessive or abusive - that is something you cannot prove or disprove numerically with statistics. We rely on the Reviews Reporting thread to high-light such cases if they arise (and there haven't been any to my knowledge)

I am sure that such an insignificant percentage is far from being abusive, I believe it's too small maybe.

the numbers are irrelevant when talking about abusive usage. If the number of 1-star reviews by an SC was 99% it still would not prove abusive use. Abusive use is when the SC gives a 1-star review to an album to deliberately lower the rating - I DO NOT BELIEVE SUCH A PRACTISE EXISTS, and even if it were 1 review out of a 1000, it would still be wrong, however you cannot prove or disprove it with numbers. You cannot gauge the intent of the reviewer by counting the number of 1-star reviews.

Ivan_Melgar_M wrote:

dean wrote:

Not that any one actually claimed either of those points - the point being made was that a 1-star rating by an SC can excessively affect the average rating of an album if the total number of non-collab ratings is low and if it is at odds with the other ratings.

Yes, but SC's are not using this ratuing very often, I'm sure that if you get a percentage of non Collaborators, the 1 star ratuings will be uch joigher and oif you go to ratings without reviews, the number will be even higher.

So if we must worry about someone, it's not about the regular collaborators,. because if we make a low rating review even if it's supported by a strong argument, people will point their finger at us, if you don't believe me, then explain me why my Opeth review was so attacked..

A lurker that rates without reviews simply gives 1 stars as candies in Halloween, without having to explain a thing.

I think that is a fair assumption.

The weighting system goes some way towards countering the 'anonymous' fan-boy use of 1-star and 5-star ratings - unfortunately, as a down-side it does mean that ratings-with-review and Collab-reviews are more accountable because of it. That is the 'price' you pay for a weighting.

I'll not get involved into why your Still Life review was attacked, but if you have read any of the Opeth discussion threads then it should have come as no surprise to you

Ivan_Melgar_M wrote:

dean wrote:

NotAProgHead's example of the Doors box-set does demonstrate that - if the two ratings had equal weight the average would have been 2.5 - since the SC has a 3.333 times heavier weighting the result is reduced to 1.69; therefore the effect of the SC rating is dramatic such that another 3 non-SC 4-star reviews would be needed to counter the weighting (not the rating). I don't believe NAPH is complaining about the 1-star review, only the weighting.

That's why we have tha t privilege, because people know and uidebntify us, and the small percentage of 1 star ratings in SC's, proves my point.

No- I do not believe the low number of 1-star SC ratings does prove that - all it shows is that SCs (and other collabs) are more discerning in what they review. I would not imagine that SCs give the weighting a second thought when rating an album - I know I don't.

Ivan_Melgar_M wrote:

BTW: I agree with The Doors rating and I gave my arguments, I don't believe that's abusive, that collection is a rip off IMO.

Iván

It isn't abusive and nobody said it was - however, some people do not think it is a fair system for albums with a low number of reviews that is all.

What?

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 07 2008 at 19:39

Dean wrote:

Then what did you mean by stating: "How many genres have 97.54% of good albums?"?

That for any determined gruup as the Spécial Collaborators, to find only 2.46% of one star ratings sounds very small to me.

Lets assume something, 1 star means a bad albuum FOR ANY REVIEWER, i doesn't mean it's intrinsecally bad album, but if a normal Special Collabotrators finds only 2.46% of bad albums, it's very small.

This doesn't mean the album is bad per se, only that from each 100 albums a normal SC rates, finds only 2.46 bad albums.

2.46% shows that the average number of 1-star reviews by an SC is a low number - it does not disprove that the use of a 1-star rating by an SC is excessive or abusive - that is something you cannot prove or disprove numerically with statistics. We rely on the Reviews Reporting thread to high-light such cases if they arise (and there haven't been any to my knowledge)

I am sure that such an insignificant percentage is far from being abusive, I believe it's too small maybe.

Not that any one actually claimed either of those points - the point being made was that a 1-star rating by an SC can excessively affect the average rating of an album if the total number of non-collab ratings is low and if it is at odds with the other ratings.

A lurker that rates without reviews simply gives 1 stars as candies in Halloween, without having to explain a thing.

That's why we have tha t privilege, because people know and uidebntify us, and the small percentage of 1 star ratings in SC's, proves my point.

BTW: I agree with The Doors rating and I gave my arguments, I don't believe that's abusive, that collection is a rip off IMO.

Iván

Hirgwath

Forum Senior Member

Joined: July 16 2007
Location: United States
Status: Offline
Points: 262

Posted: October 07 2008 at 19:29

Just a general comment: Collab reviews can be, and more often than not are, just as unoriginal, pointless, or badly written as any user's.

I don't want to cite any examples of such bad writing, at the risk of hurting someone's feelings, so let me give an example of a *good* reviewer: Certif1ed. He backs his opinions up with loads of evidence, and so even when I disagree with him (i.e. 1 star for the most famous GYBE! album), I see where he's coming from.

Certif1ed is, however, exceptional. This isn't really all that troubling...my broader point is that there are only a few reviews worth reading anyway, and it's not at all dependent on collab status whether someone makes an intelligent case or not. Most reviews on this site consist of a few sentences of opinion, rather than analysis, history, etc.

Skwisgaar Skwigelf: taller than a tree.

Toki Wartooth: not a bumblebee.

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 07 2008 at 19:13

Ivan_Melgar_M wrote:

Dean wrote:

Ivan_Melgar_M wrote:

Having heard so many albums here, I believe the average of 1 star ratings is incredibly low, epople has done an effort not to rate too low, because there are more than 2.46% of bad albums.

Just asj yourself a question.......How many genres have 97.54% of good albums?

Iván

It represents the average number of 1-star reviews out of the average number of reviews per collab - or 270 albums each. Considering we have 19,060 albums listed in the PA, 270 albums only represents 1.42% of the database - which is a very small sample and cannot be regarded as representative of the whole archive...

Doubly so since it is a selective sample... since each reviewer did not review a unique list of albums, with some albums being represented many times (potentially 43 of those 1-stars could be for the same album).

No Dean, I0'm not trying to prove how many 1 star albums there are, it's only an exact representation of SPECIAL COLLABORATORS rating albums with one star.

Only made it about SC's because people was complaining about SC taking down the albums with excessive and abusive 1 star ratings.

And this is proved wrong, as average, each SC has 2.46% of one strar ratings which is very low.

BTW: Corrected Rico's numbers and may be slight differences in each SC, because this list has a week.

Iván

Then what did you mean by stating: "How many genres have 97.54% of good albums?"?

What?

The T

Special Collaborator

Honorary Collaborator

Joined: October 16 2006
Location: FL, USA
Status: Offline
Points: 17493

Posted: October 07 2008 at 18:41

^Yes, for example, my 1-star percentage grows smaller every week... I add about 4-5 reviews each week, and it's very difficult to come about 1-star albums... It's actually non-collabs who love to give an album by an artist they don't like 1 star, or fanboyish-ly give 5 to one they love... There must be a reasons some people are SC;s....

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 07 2008 at 17:58

Dean wrote:

Ivan_Melgar_M wrote:

Having heard so many albums here, I believe the average of 1 star ratings is incredibly low, epople has done an effort not to rate too low, because there are more than 2.46% of bad albums.

Just asj yourself a question.......How many genres have 97.54% of good albums?

Iván

No Dean, I0'm not trying to prove how many 1 star albums there are, it's only an exact representation of SPECIAL COLLABORATORS rating albums with one star.

Only made it about SC's because people was complaining about SC taking down the albums with excessive and abusive 1 star ratings.

And this is proved wrong, as average, each SC has 2.46% of one strar ratings which is very low.

BTW: Corrected Rico's numbers and may be slight differences in each SC, because this list has a week.

Iván

Edited by Ivan_Melgar_M - October 07 2008 at 18:00