Forum Home Forum Home > Site News, Newbies, Help and Improvements > Report bugs here
  New Posts New Posts RSS Feed - Ratings calculation
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic ClosedRatings calculation

 Post Reply Post Reply Page  <123>
Author
Message
Rivertree View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Band Submissions

Joined: March 22 2006
Location: Germany
Status: Offline
Points: 17627
Direct Link To This Post Posted: July 11 2007 at 15:33
Originally posted by Easy Livin Easy Livin wrote:

I think, and I'm speaking from shaky ground hereEmbarrassed, that the rating shown should not be seen as a straight average of the ratings posted.
 
It is a figure calculated by taking the rating or ratings of the album, running them through an algorithm which takes other factors into account, an coming up with a result. This result allows the album to find its relative position among all the others.
 
I know it looks strange that the figure shown for one album is not the same as the only rating for that album, but it reflects all the other constituent parts of the algorithm too.


I'm quite sure the calculation was OK in the past.
So there must have been changes during the last days which have forced this dilemma.
Maybe there is a chance to revert to a former state ...




Edited by Rivertree - July 11 2007 at 15:33


Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 16:07
Originally posted by Andrea Cortese Andrea Cortese wrote:

Originally posted by Rivertree Rivertree wrote:

WOW! It looks like the PA server has its own ratings for the music.
So this must be a breakthrough with the development of ARTIFICIAL INTELLIGENCE.

Clap  Congratulations!  Clap

It would be interesting to know who is feeding the server with the songs?

BTW - The calculation is raising the results (dynamically??) - yesterday 3.45 - today 3.82




 
LOLLOL
 
Prog reviewers has been fired!!!
 
The site does not need them anymore!


Hey... "2001 Space Odissey" computer is here!!! That sense has give evaluations to the album if then...  Already if then a 4 becomes a 2,98 (I wrote a number at random)!!!  Then I give to all 1 star and we end it here!!!
Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 16:09
Unfortunately change my manner of to appraise the album!!!  
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 11 2007 at 16:15
There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge.
Back to Top
avestin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: September 18 2005
Status: Offline
Points: 12625
Direct Link To This Post Posted: July 11 2007 at 16:36
Originally posted by Easy Livin Easy Livin wrote:

There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge.
 
 
Bob, does this change apply to albums that have not been rated as well?
 
No ratings done by anyone it says and yet there's an average rating for this album.
 
Back to Top
Rivertree View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Band Submissions

Joined: March 22 2006
Location: Germany
Status: Offline
Points: 17627
Direct Link To This Post Posted: July 11 2007 at 16:55
Originally posted by Easy Livin Easy Livin wrote:

There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge.


I can't find any thread in the Prog Music Lounge dealing with this problem Confused


Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 17:16
Originally posted by Rivertree Rivertree wrote:

Originally posted by Easy Livin Easy Livin wrote:

There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge.


I can't find any thread in the Prog Music Lounge dealing with this problem Confused
 
 
However, that thread deals with the relative positions of the album in the Top 100, not how the weighted ratings are calculated.
 
Out of (now gnawing) curiosity, I would like to see the algorithm that is used to calculate each rating.
What?
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 11 2007 at 17:25
M@x provided a link somewhere (I thought it was in the PML thread) with a link to details of the algorithm. Don't know about the album with no reviews off hand. Logic dictates it should not have any sort of calculated rating.
Back to Top
Logan View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Site Admin

Joined: April 05 2006
Location: Vancouver, BC
Status: Offline
Points: 35801
Direct Link To This Post Posted: July 11 2007 at 17:34
It's more than one album.  1309 albums in the top 4000 all-genres alone by my count last night. As I wrote in that other thread:

I wonder why do all the unrated albums in the top 4000 now have a value of 3.86 (by my fatigued checking)?  I have a theory, but...

I don't find that useful.  Taken from the top 4000 most popular albums (CLICK): Every non-rated album in the top 4000 is given a rating of 3.86 and they all fall between 1224 and 2533 (they are surrounded by reviewed albums).  Between 2882 and 3286 all the albums have 2 ratings and are given 3.83. Between 2881 and 3153, the first albums only rated once are found, and are given 3.84. And at 3379 to 4000 are all the other albums given one star - at 3.83.  All of the albums only rated three times are given a score of 3.85, 3.84, or 3.83.

The album at 4000 with one review/rating of 4 by Sean Trane which gets a list rating of 3.83.  I see looking at the Kevin Ayers page that the album ratings do vary slightly between the albums only rated once... from 3.79 to 3.83....
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 17:41
Originally posted by Easy Livin Easy Livin wrote:

M@x provided a link somewhere (I thought it was in the PML thread) with a link to details of the algorithm. Don't know about the album with no reviews off hand. Logic dictates it should not have any sort of calculated rating.
 
M@x's link pointed to a Wikipedia page on the general principle of weighted averages and another link later on pointed to the comment at the top of the Top 100 page:
 
Here is some details about how we calculate the average rating of an album and the rank of an album.
  • Average rating of an album: The classic calculation of the average but more weight is affected to the rating of progarchives.com collaborators and to rating with reviews.
  • Rank of an album: We use a calculation that compare an album average rating over all others albums in the site using this theory (http://en.wikipedia.org/wiki/Weighted_average#Example)
  • Key albums: We take the average number of votes of the artist's albums and list the one that are over + the ones with an average rating of more than 4 stars

Unfortunately that does not give details of the weighted algorithm itself, nor does it explain the anomalies that mentioned by Logan and others.

What?
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 11 2007 at 17:53
weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575
Direct Link To This Post Posted: July 11 2007 at 18:02
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
Hopefully these anomalies are only affecting the albums with a low number of ratings/reviews, otherwise the entire Top 100 would be completely skewed - and I think someone might notice that and make a bit of a fuss. Wink
What?
Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 18:05
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
Back to Top
avestin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: September 18 2005
Status: Offline
Points: 12625
Direct Link To This Post Posted: July 11 2007 at 18:09
Originally posted by Mandrakeroot Mandrakeroot wrote:

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
 
 
Anomaly, yes... or a fine example of Artificial inteligence... Although judging by some of the albums that got a rating, an inteligence with not so good taste in music....
 
 
Back to Top
Mandrakeroot View Drop Down
Forum Senior Member
Forum Senior Member

Italian Prog Specialist

Joined: March 01 2006
Location: San Foca, Friûl
Status: Offline
Points: 5851
Direct Link To This Post Posted: July 11 2007 at 18:12
Originally posted by avestin avestin wrote:

Originally posted by Mandrakeroot Mandrakeroot wrote:

Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile


If one album have zero ratings... The average is zero. Instead they come appraised also those albums.  For me a serious anomaly!!!
 
 
Anomaly, yes... or a fine example of Artificial inteligence... Although judging by some of the albums that got a rating, an inteligence with not so good taste in music....
 
 


Well... Is this the Artificial intelligence of the "2002 Space Odissey"'s computer, maybe? EmbarrassedConfusedEmbarrassedConfusedShocked
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 12 2007 at 04:02
Originally posted by MikeEnRegalia MikeEnRegalia wrote:

weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution.Smile
 
Chers Mike, I've passed on your offer to M@x!Thumbs%20Up
Back to Top
M@X View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Co-founder, Admin & Webmaster

Joined: January 29 2004
Location: Canada
Status: Offline
Points: 4028
Direct Link To This Post Posted: July 12 2007 at 07:21
A major recalculations was made in the past hours, the Average Rating of a particular album is still the same though. And therefore should have fix the 1 rating issues and others , please get back to me ...

Now, the difference is in the TOP 100 list ...

Before : Log N order by
Now: Weighted Avg, comparing an album to all others album in the site

The WR of an album is calculated like this:

I weight the average number of votes and avg rating of albums against the average rating of a specific album.

Every time a review/rating is posted I recalculate the WR for all the albums.

Like this:

    SELECT @avg_num_votes = AVG(n_ratings) FROM CD
    SELECT @avg_ratings    = AVG(CONVERT(FLOAT,rate)) FROM Rating
   
    UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings)



and the weighted_ratings is used for the position in the TOP 100.

What do you think ?
Mike ?


Max


Edited by M@X - July 12 2007 at 07:23
Prog On !
Back to Top
MikeEnRegalia View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator

Joined: April 22 2005
Location: Sweden
Status: Offline
Points: 21151
Direct Link To This Post Posted: July 12 2007 at 09:01
How about this:

SELECT @avg_num_votes = AVG(n_ratings) FROM CD WHERE n_ratings > 0
SELECT @avg_ratings   = AVG(avg_ratings) FROM CD WHERE n_ratings > 0 AND avg_ratings IS NOT NULL
   
UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings) WHERE n_ratings > 0
UPDATE CD set weighted_ratings = NULL WHERE n_ratings IS NULL OR n_ratings <= 0


I think it's important to make sure that empty CD entries (0 ratings) are excluded - the WHERE clauses make sure that this is the case. I also based the calculation of @avg_ratings on the computed average of the album - it should have a better performance since there are fewer rows to look at, and my guess is that that value also includes the reviewer weights which makes it a more accurate basis for the formula.

Finally I added a WHERE clause to the UPDATE statement and a second UPDATE because for albums without ratings the calculation is not valid: (N*R+n*r)/(N+n) = N*R/N = R for n=0 and r=0 (in other words: Albums without reviews currently get assigned the average rating of all the reviews as their weighted rating).


I hope this is helpful to you, M@x ... let me know if you have further questions!Smile




Edited by MikeEnRegalia - July 12 2007 at 09:17
Back to Top
progressive View Drop Down
Forum Senior Member
Forum Senior Member


Joined: October 08 2005
Location: Finland
Status: Offline
Points: 366
Direct Link To This Post Posted: July 13 2007 at 11:02
This is f*cked system, cannot work this way,,, and people fixing some random ratings...

weighted rating averages can remain, but only if we take the absolute average with it.

People can check themselves who many ratings some albums have and whether a Reviewer has rated it.

We can't go with the popular system because so many have only few ratings.

AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngry AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngry AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryv AngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryAngryvDead


And where's the function who all is being calculated??

I think there could be many was to calculate things (for example three) but they should be shown clearly.
And it would be nice if they were in tables that can be modified,, i mean, that the list could be ordered in many ways, for example by year or rating


Edited by progressive - July 13 2007 at 11:06

► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄
Back to Top
Easy Livin View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: February 21 2004
Location: Scotland
Status: Offline
Points: 15585
Direct Link To This Post Posted: July 13 2007 at 11:43
What's the point in being angry about it?
Back to Top
 Post Reply Post Reply Page  <123>

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.445 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.