Forum Home Forum Home > Site News, Newbies, Help and Improvements > Report bugs here
  New Posts New Posts RSS Feed - Ratings calculation
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic ClosedRatings calculation

 Post Reply Post Reply Page  123>
Author
Message Reverse Sort Order
Wutu Banale View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: October 11 2006
Location: Finland
Status: Offline
Points: 104
Direct Link To This Post Topic: Ratings calculation
    Posted: July 17 2007 at 15:33
This new system isn't good at all. The average for the albums are all screwed up and all. And the new thingy made TaaB go number 1. A spot it doesn't deserve Tongue
Käsittämättömän käsittämätöntä
Suurta ja arvostettavaa.
Back to Top
XPEHOPE3KA View Drop Down
Forum Newbie
Forum Newbie


Joined: July 13 2007
Location: R
Status: Offline
Points: 29
Direct Link To This Post Posted: July 15 2007 at 11:08

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

^ So you think that with two 5 star ratings the album should have a "score" of 5.0 and be listed above long standing albums with 4.x averages?

That's the key point! Leave the algorithm for ratings as it was (it took into account who gave the review and how "long" it was).
But when you need to list the albums anywhere... It is this place where to use an algorithm which takes the number of reviews into account as well.

So the ratings would remain in a state people got used to already (when a collab rate 5, and a noncollab rate 4 the average used to be equal to 4.60 or near - I guess, it was Ok for most visitors), and least reviewed albums won't pop up on tops of rating-lists or something.

What's done now is going to make most popular albums even more popular and the least popular albums would receive even less attention. Visitors will just look at the rating (which is now modified by a number-of-ratings-dependent algorithm, but how a visitor would now THAT??) and see it is 3.89 or so - would they be willing to get it?? It doesn't attract even to read the reviews!

Hope, you got my point of view: the current algorithm scheme is not that evident to an average visitor (it just isn't natural) and surely would make some potentially good albums unattractive.

Edited by XPEHOPE3KA - July 15 2007 at 11:09
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 15 2007 at 09:36
Originally posted by progressive progressive wrote:

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

^ So you think that with two 5 star ratings the album should have a "score" of 5.0 and be listed above long standing albums with 4.x averages?


Yes, and no. It isn't listed above them, at least if there's tow ways to list them and users could choice the way. And the previous system worked well.

And there's always top 100 (or 500) lists. There are lists for genres also. Why not for artists? It is only calculating, not taking too much resources.

It's actually next to impossible to compare two artists numerically based on album ratings. If you happen to have an easy formula for this then by all means tell me - I'd be happy to implement it on my website!

It is very odd that you must look into each album's ratings trying to calculate yourself the absolute  average.

I agree - the absolute average should be shown on the album page.

And who says reviewers' ratings are better? Or that the album is better when there's more ratings. I think proggers are quite intelligent people and because there's so few ratings in many albums, we should take them seriously and not "banning" non-reviewers or single voices.

It's not a question of being "better", but of being more reliable. And the more ratings, the more different voices the average is based on which again makes it more reliable. Of course there can always be abuse, that goes without saying.

And whatabout situation when there's album with three 4-star votes, one with two five-star votes? And Im still not sure if it is a weighting factor whether you are reviewer or not.... obscure. Very obscure things.

Can't go on like this.

Dying.

And on your tombstone it would say "why o why wasn't the PA top 100 ranking better explained to me". LOL Come on, get a life!
Back to Top
progressive View Drop Down
Forum Senior Member
Forum Senior Member


Joined: October 08 2005
Location: Finland
Status: Offline
Points: 366
Direct Link To This Post Posted: July 15 2007 at 07:40
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

^ So you think that with two 5 star ratings the album should have a "score" of 5.0 and be listed above long standing albums with 4.x averages?


Yes, and no. It isn't listed above them, at least if there's tow ways to list them and users could choice the way. And the previous system worked well.

And there's always top 100 (or 500) lists. There are lists for genres also. Why not for artists? It is only calculating, not taking too much resources.

It is very odd that you must look into each album's ratings trying to calculate yourself the absolute  average.  And who says reviewers' ratings are better? Or that the album is better when there's more ratings. I think proggers are quite intelligent people and because there's so few ratings in many albums, we should take them seriously and not "banning" non-reviewers or single voices.

And whatabout situation when there's album with three 4-star votes, one with two five-star votes? And Im still not sure if it is a weighting factor whether you are reviewer or not.... obscure. Very obscure things.

Can't go on like this.

Dying.

► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 14 2007 at 06:53
^ So you think that with two 5 star ratings the album should have a "score" of 5.0 and be listed above long standing albums with 4.x averages?

The weighted average simply means that new albums start with the combined average of all the albums in the database and then slowly work their way up.

I like it, but I agree that on the website there should be an explanation of how the algorithm works, so that people don't think that it's simply broken.
Back to Top
progressive View Drop Down
Forum Senior Member
Forum Senior Member


Joined: October 08 2005
Location: Finland
Status: Offline
Points: 366
Direct Link To This Post Posted: July 14 2007 at 06:48
It makes my life a lot more difficult!!

Though, i love and hate everything at the same time...

http://www.progarchives.com/album.asp?id=13410 there's 3,77 for example, with two 5-stars.

Maybe we should vote about it. Why are rating averages changed??

► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 13 2007 at 11:43
What's the point in being angry about it?
Back to Top
progressive View Drop Down
Forum Senior Member
Forum Senior Member


Joined: October 08 2005
Location: Finland
Status: Offline
Points: 366
Direct Link To This Post Posted: July 13 2007 at 11:02
This is f*cked system, cannot work this way,,, and people fixing some random ratings...

weighted rating averages can remain, but only if we take the absolute average with it.

People can check themselves who many ratings some albums have and whether a Reviewer has rated it.

We can't go with the popular system because so many have only few ratings.

AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngry AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngry AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryv AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryvDead


And where's the function who all is being calculated??

I think there could be many was to calculate things (for example three) but they should be shown clearly.
And it would be nice if they were in tables that can be modified,, i mean, that the list could be ordered in many ways, for example by year or rating


Edited by progressive - July 13 2007 at 11:06

► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 12 2007 at 09:01
How about this:

SELECT @avg_num_votes = AVG(n_ratings) FROM CD WHERE n_ratings > 0
SELECT @avg_ratings   = AVG(avg_ratings) FROM CD WHERE n_ratings > 0 AND avg_ratings IS NOT NULL
   
UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings) WHERE n_ratings > 0
UPDATE CD set weighted_ratings = NULL WHERE n_ratings IS NULL OR n_ratings <= 0


I think it's important to make sure that empty CD entries (0 ratings) are excluded - the WHERE clauses make sure that this is the case. I also based the calculation of @avg_ratings on the computed average of the album - it should have a better performance since there are fewer rows to look at, and my guess is that that value also includes the reviewer weights which makes it a more accurate basis for the formula.

Finally I added a WHERE clause to the UPDATE statement and a second UPDATE because for albums without ratings the calculation is not valid: (N*R+n*r)/(N+n) = N*R/N = R for n=0 and r=0 (in other words: Albums without reviews currently get assigned the average rating of all the reviews as their weighted rating).


I hope this is helpful to you, M@x ... let me know if you have further questions!Smile




Edited by MikeEnRegalia - July 12 2007 at 09:17
Back to Top
M@X View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Co-founder, Admin & Webmaster

Joined: January 29 2004
Location: Canada
Status: Offline
Points: 4028
Direct Link To This Post Posted: July 12 2007 at 07:21
A major recalculations was made in the past hours, the Average Rating of a particular album is still the same though. And therefore should have fix the 1 rating issues and others , please get back to me ...

Now, the difference is in the TOP 100 list ...

Before : Log N order by
Now: Weighted Avg, comparing an album to all others album in the site

The WR of an album is calculated like this:

I weight the average number of votes and avg rating of albums against the average rating of a specific album.

Every time a review/rating is posted I recalculate the WR for all the albums.

Like this:

    SELECT @avg_num_votes = AVG(n_ratings) FROM CD
    SELECT @avg_ratings    = AVG(CONVERT(FLOAT,rate)) FROM Rating
   
    UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings)



and the weighted_ratings is used for the position in the TOP 100.

What do you think ?
Mike ?


Max


Edited by M@X - July 12 2007 at 07:23
Prog On !
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 12 2007 at 04:02
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
 
Chers Mike, I've passed on your offer to M@x!Thumbs%20Up
Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 18:12
Originally posted by avestin avestin wrote:

Originally posted by Mandrakeroot Mandrakeroot wrote:

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
 
 
Anomaly, yes... or a fine example of Artificial inteligence... Although judging by some of the albums that got a rating, an inteligence with not so good taste in music....
 
 


Well... Is this the Artificial intelligence of the "2002 Space Odissey"'s computer, maybe? EmbarrassedConfusedEmbarrassedConfusedShocked
Back to Top
avestin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: September 18 2005
Status: Offline
Points: 12625
Direct Link To This Post Posted: July 11 2007 at 18:09
Originally posted by Mandrakeroot Mandrakeroot wrote:

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
 
 
Anomaly, yes... or a fine example of Artificial inteligence... Although judging by some of the albums that got a rating, an inteligence with not so good taste in music....
 
 
Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 18:05
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 18:02
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
Hopefully these anomalies are only affecting the albums with a low number of ratings/reviews, otherwise the entire Top 100 would be completely skewed - and I think someone might notice that and make a bit of a fuss. Wink
What?
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 11 2007 at 17:53
weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 17:41
Originally posted by Easy Livin Easy Livin wrote:

M@x provided a link somewhere (I thought it was in the PML thread) with a link to details of the algorithm. Don't know about the album with no reviews off hand. Logic dictates it should not have any sort of calculated rating.
 
M@x's link pointed to a Wikipedia page on the general principle of weighted averages and another link later on pointed to the comment at the top of the Top 100 page:
 
Here is some details about how we calculate the average rating of an album and the rank of an album.
  • Average rating of an album: The classic calculation of the average but more weight is affected to the rating of progarchives.com collaborators and to rating with reviews.
  • Rank of an album: We use a calculation that compare an album average rating over all others albums in the site using this theory (http://en.wikipedia.org/wiki/Weighted_average#Example)
  • Key albums: We take the average number of votes of the artist's albums and list the one that are over + the ones with an average rating of more than 4 stars

Unfortunately that does not give details of the weighted algorithm itself, nor does it explain the anomalies that mentioned by Logan and others.

What?
Back to Top
Logan View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Site Admin

Joined: April 05 2006
Location: Vancouver, BC
Status: Online
Points: 35795
Direct Link To This Post Posted: July 11 2007 at 17:34
It's more than one album.  1309 albums in the top 4000 all-genres alone by my count last night. As I wrote in that other thread:

I wonder why do all the unrated albums in the top 4000 now have a value of 3.86 (by my fatigued checking)?  I have a theory, but...

I don't find that useful.  Taken from the top 4000 most popular albums (CLICK): Every non-rated album in the top 4000 is given a rating of 3.86 and they all fall between 1224 and 2533 (they are surrounded by reviewed albums).  Between 2882 and 3286 all the albums have 2 ratings and are given 3.83. Between 2881 and 3153, the first albums only rated once are found, and are given 3.84. And at 3379 to 4000 are all the other albums given one star - at 3.83.  All of the albums only rated three times are given a score of 3.85, 3.84, or 3.83.

The album at 4000 with one review/rating of 4 by Sean Trane which gets a list rating of 3.83.  I see looking at the Kevin Ayers page that the album ratings do vary slightly between the albums only rated once... from 3.79 to 3.83....
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 11 2007 at 17:25
M@x provided a link somewhere (I thought it was in the PML thread) with a link to details of the algorithm. Don't know about the album with no reviews off hand. Logic dictates it should not have any sort of calculated rating.
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 17:16
Originally posted by Rivertree Rivertree wrote:

Originally posted by Easy Livin Easy Livin wrote:

There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge.


I can't find any thread in the Prog Music Lounge dealing with this problem Confused
 
 
However, that thread deals with the relative positions of the album in the Top 100, not how the weighted ratings are calculated.
 
Out of (now gnawing) curiosity, I would like to see the algorithm that is used to calculate each rating.
What?
Back to Top
 Post Reply Post Reply Page  123>

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.465 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.