Forum Home Forum Home > Progressive Music Lounges > Prog Polls
  New Posts New Posts RSS Feed - Collaborators’ TOP 20
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic ClosedCollaborators’ TOP 20

 Post Reply Post Reply Page  <12
Poll Question: Should a rating without a review count in deriving the TOP 20?
Poll Choice Votes Poll Statistics
4 [19.05%]
17 [80.95%]
This topic is closed, no new votes accepted

Author
Message
Fitzcarraldo View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 30 2004
Location: United Kingdom
Status: Offline
Points: 1835
Direct Link To This Post Posted: September 10 2004 at 16:46

Easy Livin,

If you look at my original suggestion for the TOP 20 (see hyperlink below) you'll see that I discussed the algorithm and mentioned the problem of extreme cases.

http://www.progarchives.com/forum/forum_posts.asp?TID=697

One of the reasons I ruled out just using averages was for the reason you point out. (Another reason is that it does not allow differentiation between the hundreds of albums that would have the same average rating.)

If you look at "Train Of Thought" you'll see that it *is* popular:

33 5-star ratings; 9 4-star ratings; 11 3-star ratings; 5 2-star ratings; 8 1-star ratings.

Compare that with "Dark Side Of The Moon":

28 5-star ratings; 2 4-star ratings; 3 3-star ratings; 1 2-star rating.

So, from the above, to me "Train Of Thought" *is* more popular than DSOTM whichever way you look at it. And addressing your comment about the number of people who have rated it, the fact that more people have rated it is, in itself, a possible indication of popularity.

When I checked the "real" TOP 50 list in June, it was a good reflection of popularity, i.e. the 'extreme case' scenario seems not to occur in practice.

 

Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: September 10 2004 at 17:37

Fitzcarraldo,

It's fun isn't it!Big smile

I don't think any measure which uses the total number of "stars" as its main measure is particulary valid. In the example you quote, TOT is rated by over a third of the people as only "good"(3 stars) or worse, hardly an indication that it is universally liked. I would not like to think that if I then gave it two stars, I was making it appear more popular. (Hypothetical by the way, I have not yet heard TOT.) We will all have contributed reviews and ratings for albums we did not enjoy much. The fact that an album has a lot of ratings does not in itself make it "popular". There are quite a few albums with a lot of ratings, the majority of which are adverse. (Yes and Genesis have a few each).

I think averaging is the best solution, but with the safeguard that the album must have a minimum number of reviews (say 10) to qualify. I do acknowledge though that that too is flawed. Perhaps the star ratings is not the best way to identify the top albums. Maybe there could be a separate voting section for everyone's top 10. Registered voters (i.e. forum members) could change their top 10 at any time. That would immediately remove the adverse star ratings from the calculation, and thus focus on the positive. It would also mean that if I try to differentiate between two great albums by the same band by giving one 4 stars, and one 5 stars, I can still support both in my top 10 vote.

By he way, taking a straight averaging of the star ratings, DSOTM comes out far higher (4.67 vs 3.81)

TOT

Star rating

Number of votes

Total points

5

33

165

4

9

36

3

11

33

2

5

10

1

8

8

Totals

66

252

Average points

3.81

DSOTM

Star rating

Number of votes

Total points

5

28

140

4

2

8

3

3

9

2

1

2

1

0

0

Totals

34

159

Average points

4.67

Back to Top
Fitzcarraldo View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 30 2004
Location: United Kingdom
Status: Offline
Points: 1835
Direct Link To This Post Posted: September 10 2004 at 18:04

Fun, it is!

But, as I think we both agree, the comparison of averages is problematic:

Album 1 has 20 5-star ratings => average = 5-stars

Album 2 has 60 4-star ratings => average = 4-stars

Which of the above two albums is more popular?

I think the number of reviews needs to be a factor. And I think that the more people/reviews are included, the better (i.e. the bigger the sample, the more interesting and representative the result).

I need to think some more about this (interesting) problem!

Back to Top
M@X View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Co-founder, Admin & Webmaster

Joined: January 29 2004
Location: Canada
Status: Offline
Points: 4028
Direct Link To This Post Posted: September 12 2004 at 09:01

Prog Fans ,

This is cool to have great polls, manage and fill with bright POSTS like thoses in this forum. That is why the FORUM of PROG ARCHIVES is such important for us.

Continue your great work.

About the TOP 20 Collaborator's album on the HOME PAGE , as requested and debated in this THREAD, the algorithm now ( 09-12-2004 )  count only ratings of collaborators with review with 50 caracters and more.

I think it is better now, but not a lot of changes...

Next question :

"Should we count all ratings by collaborators and non-collaborators ?"


MAX@

 

 



Edited by MAX@
Prog On !
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: September 12 2004 at 10:37
Originally posted by MAX@ MAX@ wrote:

"Should we count all ratings by collaborators and non-collaborators ?"


MAX@

I still say no to that one Max. I know at first it seems a bit elitist, but I retain my concerns about spamming when it becomes known that all votes count. (And rest assured it would, the bulletin boards on the various band sites would soon have notices on them saying "vote for our band"). Until we collecively come up with a reasonably foolproof way of doing it, I say stick with what we have.Thumbs Up

Back to Top
James Lee View Drop Down
Special Collaborator
Special Collaborator

Honorary Collaborator

Joined: June 05 2004
Status: Offline
Points: 3525
Direct Link To This Post Posted: September 12 2004 at 15:50
I think we should only count ratings from andrea salvador.
Back to Top
Fitzcarraldo View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 30 2004
Location: United Kingdom
Status: Offline
Points: 1835
Direct Link To This Post Posted: September 12 2004 at 16:05

Thanks, MAX@.

Well, regarding your second question, I think my opinion is known, but I also understand Easy Livin's concerns.

Is there some way we could protect against the spamming that Easy Livin thinks may be a possibility? For example, only include star-ratings from non-Collaborators if there is an associated review of at least a certain length? Only disclose the results to members of the forums?

Whatever is done, I think it would still be useful to have the Collaborators' TOP 20/50 as a separate list, if for no other reason than to give some recommendations to newcomers from the 'official' site reviewers. But as we've seen in this thread, the Collaborators tastes do not appear to be an indication of the tastes of the larger population.

When I get time I'm going to think about the algorithm. But I'm convinced that averaging is not the best way to do it: it does not make sense to me if an album has, say, ten 5-star ratings (average = 5 stars) and then someone adds a 4-star rating (average = 4.91) and the album drops in the ratings. Easy Livin might counter by saying that adding e.g. a 2-star review should not raise an album in the ratings, and he might also be right. But if Album1 has more 5-star and 4-star reviews than Album2 (as in the example I quoted in an earlier posting) then, to me, even if Album1 has more 2-star and below ratings, Album1 is the more popular. I say this because more people like Album1 than Album2, simple as that. But I'll give the algorithm some more thought.

 

Back to Top
richardh View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: February 18 2004
Location: United Kingdom
Status: Online
Points: 28023
Direct Link To This Post Posted: September 12 2004 at 16:12
ELP have now dropped out of the top twenty.
Back to Top
threefates View Drop Down
Forum Senior Member
Forum Senior Member


Joined: June 30 2004
Location: United States
Status: Offline
Points: 4215
Direct Link To This Post Posted: September 12 2004 at 16:14
Originally posted by JrKASperov JrKASperov wrote:

Originally posted by threefates threefates wrote:

I don't!!  ELP is way underrated, BSS should be in the top 10 and Trilogy is missing.  That in itself is a sin!!

No.

No what?

THIS IS ELP
Back to Top
threefates View Drop Down
Forum Senior Member
Forum Senior Member


Joined: June 30 2004
Location: United States
Status: Offline
Points: 4215
Direct Link To This Post Posted: September 12 2004 at 16:17

Originally posted by richardh richardh wrote:

ELP have now dropped out of the top twenty.

Again Richard... its just me and you.  That is so disappointing... I'm really very disappointed with everyone here..

I think I'm gonna go play some Trilogy now to raise my spirits..

THIS IS ELP
Back to Top
Fitzcarraldo View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 30 2004
Location: United Kingdom
Status: Offline
Points: 1835
Direct Link To This Post Posted: September 12 2004 at 16:30
I haven't reviewed my ELP albums yet!
Back to Top
JrKASperov View Drop Down
Forum Senior Member
Forum Senior Member


Joined: July 07 2004
Status: Offline
Points: 904
Direct Link To This Post Posted: September 12 2004 at 17:31

Originally posted by threefates threefates wrote:

No what?

No whatever you say

Epic.
Back to Top
richardh View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: February 18 2004
Location: United Kingdom
Status: Online
Points: 28023
Direct Link To This Post Posted: September 13 2004 at 03:08
ELP back in the top twenty thanks to Ivan's review
Back to Top
 Post Reply Post Reply Page  <12

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.234 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.