too heavily weighted ratings ? its outrageous - Progressive Rock Music Forum

Author

Message

Topic Search

Topic Options

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: September 14 2008 at 18:16

^ Exactly. A one-star rating by a Prog Reviewer or a Collaborator should be an accurate assessment in their opinion and be accompanied by reasoning within the review text to justify it. But that should also hold true for any one-star rating.

What?

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: September 14 2008 at 18:53

NotAProghead wrote:

It seems discussion is drowned in details.

OK, collaborators "earned" their heavy voices, they sacrificed their lives to prog

, they never rest, never sleep etc.

Honestly, I can't care less for the ratings and teh Topp 100 list, my opinion won't change, I lobe Hybris, no matter is in the 1 or in the 20 spot, you can stay with the weight if you want, I don't care for it.

NotAProghead wrote:

But there is another side of the medal: if your voice means more than others, you should feel the responsibility. Everytime, when writing each review. Forget the words "this album stinks" and many other nice phrases, be extremely accurate with 1-star ratings. Am I wrong?

Count the number of 1 stars reviews I have for example, 10 out of 230 reviews, 4.32%...Less than 5%

6 for 3 men Genesis era albums
2 for Love Beach and Re-Works
1 for Union and one for Big Generator

Not a single controversial album, most of them rated with 1 star when it was possible to give 0 stars,

As long as I remember, I don't use phrases as This album stinks.

Never did a rating without review, or a short review tha won't appear,.

Iván

NotAProghead View Drop Down

Special Collaborator

Errors & Omissions Team

Joined: October 22 2005
Location: Russia
Status: Offline
Points: 7973

Posted: September 14 2008 at 18:58

Dean wrote:

^ Exactly. A one-star rating by a Prog Reviewer or a Collaborator should be an accurate assessment in their opinion and be accompanied by reasoning within the review text to justify it. But that should also hold true for any one-star rating.

Yes, but if non-collabs one-star rating can be easily balanced by any other rating, 1-star collaborator's rating can make a "bad reputation" for an album for quite a long time.

This is why I tell of a special collaborators' responsibility.

Edited by NotAProghead - September 14 2008 at 18:59

Who are you and who am I to say we know the reason why... (D. Gilmour)

NotAProghead View Drop Down

Special Collaborator

Errors & Omissions Team

Joined: October 22 2005
Location: Russia
Status: Offline
Points: 7973

Posted: September 14 2008 at 19:03

Ivan, I did not mean your reviews. But I see, from time to time, not well-thought collabs' reviews.

Who are you and who am I to say we know the reason why... (D. Gilmour)

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: September 14 2008 at 20:11

NotAProghead wrote:

Ivan, I did not mean your reviews. But I see, from time to time, not well-thought collabs' reviews.

I took the time to check all the Special Collaborators 1 stars reviews, this are the results:

1	.-	Peter	4 out of 122
2	.-	Dan Borobowsky	3 out of 140
3	.-	Lucas	1 out of 51
4	.-	Joren	0 out of 51
5	.-	Phillippe	17 out 0f 421
6	.-	Oliverstoned	4 out of 93
7	.-	Bryan	29 out of 257
8	.-	Cert1fied	7 out of 168
9	.-	Dick Heath	4 out of 82
10	.-	Hibou	2 out of 80
11	.-	Iván	10 out of 233
12	.-	Sean Trane	134 out of 2489
13	.-	Fitzcarraldo	1 out of 110
14	.-	Seyo	32 out of 527
15	.-	Cesar Inca	0 out of 677
16	.-	Garion	3 out of 108
17	.-	Gatot	23 out of 1412
18	.-	James Lee	1 out of 137
19	.-	Chris Stacey	39 out of 634
20	.-	Trouserpress	0 out of 18
21	.-	Greenback	6 out of 685
22	.-	Man Erg	0 out of 7
23	.-	Alucard	2 out of 90
24	.-	Trotsky	12 out of 264
25	.-	Syzygy	8 out of 205
26	.-	Cygnus X-2	2 out of 353
27	.-	Ricochet	17 out of 854
28	.-	Jimbo	2 out of 143
29	.-	Snow Dog	2 out of 12
30	.-	Salmacis	6 out of 551
31	.-	MikeEnRegalia	2 out of 62
32	.-	Yanns	0 out of 40
33	.-	Tuzvihar	0 out of 4
34	.-	Memowakeman	1 out of 263
35	.-	Golden Spyral	0 out of 36
36	.-	BJ-1	0 out of 240
37	.-	Eetu	13 out of 568
38	.-	Fragile DT	0 out of 9
39	.-	Zac M	0 out of 50
40	.-	James	0 out of 10
41	.-	Chopper	2 out of 96
42	.-	Erik Neuteboom	6 out of 1202
43	.-	ClemofNazareth	27 out of 650
44	.-	Horza	3 out of 60
45	.-	Andrea Cortese	1 out of 280
46	.-	Anael	0 out of 4
47	.-	Avestin	0 out of 112
48	.-	Micky	2 out of 67
49	.-	Chamberry	0 out of 111
50	.-	E-Dub	1 out of 64
51	.-	Mystic Fred	0 out of 73
52	.-	Rivertree	1 out of 207
53	.-	Joolz	3 out of 176
54	.-	Logan	0 out of 1
55	.-	Bhikkhu	0 out of 83
56	.-	Fassbinder	0 out of 4
57	.-	Clarke 2001	14 out of 211
58	.-	Atavachron	7 out of 248
59	.-	The T	11 out of 190
60	.-	Chus	0 out of 43
61	.-	Kenethlevine	12 out of 397
62	.-	Finnforest	9 out of 490
63	.-	Burritounit	0 out of 28

Only one SC has reached 10%
Four SC have reached 5%
The rest are bellow 5%

I belive that contrary to what I expected, the SC's think it more than twice before giving a one star rating, I believe that 10% of 1 stars would be conservative, only one member has reached that percentage, 4 have reached 50% of that and 58 are bellow 5%

What less can you expect?

Iván

BTW: Too lazy to check all Prog Reviewers and Collaborators.

Edited by Ivan_Melgar_M - September 14 2008 at 20:13

debrewguy

Special Collaborator

Honorary Collaborator

Joined: April 30 2007
Location: Canada
Status: Offline
Points: 3596

Posted: September 14 2008 at 20:44

That might be the point Ivan. You folks are 'required" to justify your ratings with a review. And from my misty memory over the past two - three years here at PA, any collab/reviewer/admin that felt they weren't up to the task of doing this announced their departure or resignation. I do recall a few cases where one of PA's resident experts were called to task in the reviews discussion & reporting threads, and do not remember these posts to have been censored or derided. Indeed, in many cases, admin posted a response indicating that the reviewer had already been asked to "review" their review. AND EVEN BETTER, a few even came back to say that indeed , their review was inadequate in the sense that it did not meet PA's guidelines and also the expectations the rest of us have.
And to add to this, a few threads have asked how one becomes a collab or prog reviewer. Simply put, the answer every time was that with time, prospective candidates are spotted by admin by the quality of their reviews, and just invited to joing the inner circle. Some have even been thrown out for behaviour found unacceptable to M@x and admin.

"Here I am talking to some of the smartest people in the world and I didn't even notice,” Lieutenant Columbo, episode The Bye-Bye Sky-High I.Q. Murder Case.

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: September 14 2008 at 21:34

debrewguy wrote:

That might be the point Ivan. You folks are 'required" to justify your ratings with a review. And from my misty memory over the past two - three years here at PA, any collab/reviewer/admin that felt they weren't up to the task of doing this announced their departure or resignation.

That only proved the honesty of a member, if he doesn't feel he can give everything he's asked, the most honest thing to do is to resign.

I do recall a few cases where one of PA's resident experts were called to task in the reviews discussion & reporting threads, and do not remember these posts to have been censored or derided. Indeed, in many cases, admin posted a response indicating that the reviewer had already been asked to "review" their review. AND EVEN BETTER, a few even came back to say that indeed , their review was inadequate in the sense that it did not meet PA's guidelines and also the expectations the rest of us have.

Not censored DB, in thse cases the thread has been moved to the Collaborators sectionm, and in some cases the reviews have been deleted.

And to add to this, a few threads have asked how one becomes a collab or prog reviewer. Simply put, the answer every time was that with time, prospective candidates are spotted by admin by the quality of their reviews,

Why ask? Read the FAQ

General

How do I become a Prog Reviewer/Special Collaborator?
"Prog reviewers" are people who have been identified by the site’s owners as regularly contributing high quality reviews of a diverse range of bands and albums.

"Special collaborators" also regularly submit high quality reviews, but in addition they participate in (or have participated in) the running of the site.

You would normally become a Prog Reviewer or Special Collaborator by invitation from the site owners. If however, you feel you have met the following criteria, and would like to be considered for induction as a Prog Reviewer, send a Private Message (PM) to "Atkingani" (the Collaborator Manager) making your case.

NOTE: It's important to bear in mind that the following criteria are solely for general guidance, and do not constitute any sort of prerequisites or guarantee of acceptance. Also, that although applications can be submitted at any time, the processing of applications will be done only a few times each year, so please do not be impatient.

Submit at least 30 well composed reviews, which talk in some detail about each album. Ideally, the albums covered will be spread across a range of bands and sub-genres, and will not simply focus on those in the top 100. If possible, try to do a sequence of at least 5 albums by one of these "lesser-known" bands.
Contribute one or more band biographies for bands on the site who do not presently have biographic details. (With a little time and research, these can be prepared quite easily, even if you are not too familiar with the band. Do not however simply copy the words from other people’s work. Collate the facts, then present them in your own words.)
Participate constructively in the forum, offering intelligent contributions and avoiding becoming embroiled in personal battles with other members. (When applying, make sure to mention both your reviewer name AND forum ID).
If you can obtain the agreement of a current Prog Reviewer or Special Collaborator to support your request, that will also help.

To become a Special Collaborator, you will normally need to have been a Prog Reviewer first. Prog Reviewers who are willing to contribute in some way to the running of the site should contact Tony R giving details of the type of function you would be willing to undertake.

http://www.progarchives.com/forum/forum_posts.asp?TID=46254&KW=Collaborator

Of corse for a team member, each head can choose any member IN WHO'S CAPACITY HE TRUSTS.

and just invited to joing the inner circle. Some have even been thrown out for behaviour found unacceptable to M@x and admin.

Well, we don't have a cruystal ball to know who will become a troll after being a collaborative member. But thrown out? I don't remember a case, only demoted if I'm right.

Iván

Yorkie X

Forum Senior Member

Joined: July 04 2007
Status: Offline
Points: 1049

Posted: September 15 2008 at 03:27

^^ That's too Many one star reviews if you ask me ... but since nobody asked me I`ll shut up Stern%20Smile

The T

Special Collaborator

Honorary Collaborator

Joined: October 16 2006
Location: FL, USA
Status: Offline
Points: 17493

Posted: September 15 2008 at 14:31

Yorkie X wrote:

^^ That's too Many one star reviews if you ask me ... but since nobody asked me I`ll shut up Stern%20Smile

An average of 5% of 1 star reviews is too many? (By the way Ivan, fantastic job...

) Have in mind that, unlike non-collabs, most collabs have a LOT of reviews, usually more than a hundred. I guess your basic reasoning is that it's very difficult to find albums that are so atrocious that they deserve 1-star reviews, that even the worst albums have some redeeming features. But the fact that most of those reviews are actual REVIEWs, not just stars with no text, tells you that maybe the justification for such rating can be found there.

Also, it's collabs usually who care enough about this site to review some of their reviews and change them. I, for example, am not happy with my first 50-60 reviews as they contain lots of atrocious arguments and stupid remarks. I've changed a few. I even changed two 1-star reviews to 2 and 3 stars respectively (both Genesis albums) and other collabs are always doing similar things, adjusting checking their reviews. Because they care. And that's one of the basic points of being a collab and having your rating weigh more: you have proven you care about PA. It's not just another website you navigate on. It's something else. Call it a disfunctional-social-club (as we never meet) or whatever but it's not just a website. For most non-collabs, it IS.

I could justify each of my 11 1-star reviews. And the text would do it by itself. And I'm 101% sure that all collabs could do the same. Because... they CARE.

debrewguy

Special Collaborator

Honorary Collaborator

Joined: April 30 2007
Location: Canada
Status: Offline
Points: 3596

Posted: September 15 2008 at 18:36

Could it be that collabs and reviewers make it a point to listen to and review albums or groups outside their usual preferences ?
I don't know about most music fans, but I only buy albums that I figure I'll be interested in. And if I can sample it before , so much the better.
Or, to give you a hopefully uncontroversial example - I'm a Motorhead fan. I did not buy Snake Bite Love, and will not buy Motorizer after having listened to samples. So for two M-Head albums that I likely would have rated 1 (on a metal site), I wouldn't have rated or reviewed , 'cause I wouldn't have bothered buying them.
I know many prog acts cannot be judged based on 30 second snippets. But many fans will use some sort of reference point in deciding what they will buy. Therefore , increasing the chances that they would actually like the music. And so up goes the the odds any album would get a decent rating.
Unless you're masochistic and make it a point to pick up albums or groups that you know you will hate.

"Here I am talking to some of the smartest people in the world and I didn't even notice,” Lieutenant Columbo, episode The Bye-Bye Sky-High I.Q. Murder Case.

Einsetumadur View Drop Down

Prog Reviewer

Joined: September 24 2008
Location: Germany
Status: Offline
Points: 265

Posted: October 07 2008 at 08:13

This 10/3/1 rating system is rather sensible, a good try to make people review more albums.

But what could really disappoint new reviewers is that their reviews are really hard to find ...

RáulGuate has done a fine review of Yes' "Keystudio".
You can just read the introduction of this review on the "Keystudio" page, 50 words of this review - somewhere smushed in the corner. And when you've read all those Collaborator-reviews you do not want more reviews to this album... Big%20smile

But if you nevertheless want to read the new reviewers' reviews, all the Collaborators' reviews are put inbetween at all places.

And even albums where no collaborator has yet done a review have the other reviews pushed into the corner.

Don't get me wrong - I find it great that I can review here after just registering in the forum. But what about sorting all the reviews according to the date - without separating between 'good and bad'? Wink

Greets
Einsetumadur

Edited by Einsetumadur - October 07 2008 at 08:14

All in all each man in all men

Easy Livin View Drop Down

Special Collaborator

Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585

Posted: October 07 2008 at 12:20

M@x is considering a number of improvements to the site, the presentation of the reviews is on the list.

Ivan_Melgar_M View Drop Down

Special Collaborator

Honorary Collaborator

Joined: April 27 2004
Location: Peru
Status: Offline
Points: 19557

Posted: October 07 2008 at 13:03

Advanced chart of 1 star ratings:

1	.-	Peter	4 out of 122	3.27%
2	.-	Dan Borobowsky	3 out of 140	2.14%
3	.-	Lucas	1 out of 51	1.96%
4	.-	Joren	0 out of 51	0%
5	.-	Phillippe	17 out 0f 421	4.00%
6	.-	Oliverstoned	4 out of 93	4.30%
7	.-	Bryan	29 out of 257	11.28%
8	.-	Cert1fied	7 out of 168	4.14%
9	.-	Dick Heath	4 out of 82	4.87%
10	.-	Hibou	2 out of 80	2.50%
11	.-	Iván	10 out of 240	4.16%
12	.-	Sean Trane	134 of 2489	5.30%
13	.-	Fitzcarraldo	1 out of 110	0.90%
14	.-	Seyo	32 out of 527	6.07%
15	.-	Cesar Inca	0 out of 677	0%
16	.-	Garion	3 out of 108	2.77%
17	.-	Gatot	23 out of 1412	1.60%
18	.-	James Lee	1 out of 137	0.72%
19	.-	Chris Stacey	39 out of 634	6.15%
20	.-	Trouserpress	0 out of 18	0%
21	.-	Greenback	6 out of 685	0.67%
22	.-	Man Erg	0 out of 7	0%
23	.-	Alucard	2 out of 90	2.22%
24	.-	Trotsky	12 out of 264	4.54%
25	.-	Syzygy	8 out of 205	3.90%
26	.-	Cygnus X-2	2 out of 353	0.50%
27	.-	Ricochet	17 out of 954	1.78%
28	.-	Jimbo	2 out of 143	1.39%
29	.-	Snow Dog	2 out of 12	16.60%
30	.-	Salmacis	6 out of 551	1.06%
31	.-	MikeEnRegalia	2 out of 62	3.22%
32	.-	Yanns	0 out of 40	0%
33	.-	Tuzvihar	0 out of 4	9%
34	.-	Memowakeman	1 out of 263	0.38%
35	.-	Golden Spyral	0 out of 36	0%
36	.-	BJ-1	0 out of 240	0%
37	.-	Eetu	13 out of 568	2.88%
38	.-	Fragile DT	0 out of 9	2.00%
39	.-	Zac M	0 out of 50	0%
40	.-	James	0 out of 10	0%
41	.-	Chopper	2 out of 96	2.08%
42	.-	Erik Neuteboom	6 out of 1202	0.49%
43	.-	ClemofNazareth	27 out of 650	4.15%
44	.-	Horza	3 out of 60	5.00%
45	.-	Andrea Cortese	1 out of 280	0.35%
46	.-	Anael	0 out of 4	0%
47	.-	Avestin	0 out of 112	0%
48	.-	Micky	2 out of 67	2.98%
49	.-	Chamberry	0 out of 111	0%
50	.-	E-Dub	1 out of 64	1.56%
51	.-	Mystic Fred	0 out of 73	0%
52	.-	Rivertree	1 out of 207	0.48%
53	.-	Joolz	3 out of 176	1.70%
54	.-	Logan	0 out of 1	0%
55	.-	Bhikkhu	0 out of 83	0%
56	.-	Fassbinder	0 out of 4	0%
57	.-	Clarke 2001	14 out of 211	6.63%
58	.-	Atavachron	7 out of 248	2.80%
59	.-	The T	11 out of 190	5.70%
60	.-	Chus	0 out of 43	0%
61	.-	Kenethlevine	12 out of 397	3.02%
62	.-	Finnforest	9 out of 490	1.83%
63	.-	Burritounit	0 out of 28	0%

1.- General Average of 1 star ratings.................2.46

2.- Averages above 5.99%.....................................3

3.- Averages above 10%.........................................2

4.- Averages bellow 5.99%.....................................58

Having heard so many albums here, I believe the average of 1 star ratings is incredibly low, epople has done an effort not to rate too low, because there are more than 2.46% of bad albums.

Just asj yourself a question.......How many genres have 97.54% of good albums?

Iván

Edited by Ivan_Melgar_M - October 07 2008 at 16:50

Logan

Forum & Site Admin Group

Site Admin

Joined: April 05 2006
Location: Vancouver, BC
Status: Offline
Points: 37598

Posted: October 07 2008 at 13:37

I wonder why mine is 0 out of 1. I doubled my review count some days ago. LOL

Perhaps it's an old list, or the various category doesn't count (I'd rather offer a little review in a bio, or on the forum, anyway).

4%20stars

DUFFARD, PASCAL - Dieu est Fou (read review)
3%20stars

VARIOUS ARTISTS (TRIBUTES) - Hamtaï! Hommage à la musique de Christian Vander (read review)

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 07 2008 at 13:49

^ write a 1-star review now Greg and you'll top Iván's league table

What?

Ricochet

Special Collaborator

Honorary Collaborator

Joined: February 27 2005
Location: Nauru
Status: Offline
Points: 46301

Posted: October 07 2008 at 14:00

^I have 954, not 854 reviews. LOL

And just 17 1-star reviews is not enough. Kid-A reviewing, now!! Wink

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 07 2008 at 14:50

Ivan_Melgar_M wrote:

Having heard so many albums here, I believe the average of 1 star ratings is incredibly low, epople has done an effort not to rate too low, because there are more than 2.46% of bad albums.

Just asj yourself a question.......How many genres have 97.54% of good albums?

Iván

Ah... you can't actually do that calculation (100%-2.46%=97.54%) because 2.46% does not represent the average number of "bad" albums in the archive - or anything even related to it.

It represents the average number of 1-star reviews out of the average number of reviews per collab - or 270 albums each. Considering we have 19,060 albums listed in the PA, 270 albums only represents 1.42% of the database - which is a very small sample and cannot be regarded as representative of the whole archive...

Doubly so since it is a selective sample... since each reviewer did not review a unique list of albums, with some albums being represented many times (potentially 43 of those 1-stars could be for the same album).

What?

Dean

Special Collaborator

Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575

Posted: October 07 2008 at 14:52

Ricochet wrote:

^I have 954, not 854 reviews. LOL

And just 17 1-star reviews is not enough. Kid-A reviewing, now!! Wink

Oi! No Xover bashing Vic

What?

Ricochet

Special Collaborator

Honorary Collaborator

Joined: February 27 2005
Location: Nauru
Status: Offline
Points: 46301

Posted: October 07 2008 at 15:06

Dean wrote:

Ricochet wrote:

^I have 954, not 854 reviews. LOL

And just 17 1-star reviews is not enough. Kid-A reviewing, now!! Wink

Oi! No Xover bashing Vic

Like Radiohead, hate Kid A! Smile

My overall rating for Radiohead albums (eventually) should be positive, in total, for Crossover. Wink

limeyrob

Forum Senior Member

VIP Member

Joined: January 15 2005
Location: England
Status: Offline
Points: 1402

Posted: October 07 2008 at 15:28

I think the system is fair.

Personally I think that SCs look at it from a prog point of view. I would score according to my own taste. That's if I did much scoring. I don't think I know enough about the music progwise to score and I don't think I would review an album if I hadn't bought it beforehand.

However I do consult PA before buying to get people's views. Sometimes I'll buy based on good recommendations only to find that I didn't like it. And not just after the first play usually. Sometimes I'll like an album that didn't get very good reviews.

Sometimes I wonder why some albums are so highly rated, but that's just my taste. Also it reduces the effect of people low scoring just because they don't like the artist/genre - or vice versa. So yeah, the system works OK.

Post Reply	Page <1 4 567 8 9 >