Ratings calculation
Printed From: Progarchives.com
Category: Site News, Newbies, Help and Improvements
Forum Name: Report bugs here
Forum Description: Help us improve the site from a tech standpoint
URL: http://www.progarchives.com/forum/forum_posts.asp?TID=39782
Printed Date: December 01 2024 at 23:32
Software Version: Web Wiz Forums 11.01 - http://www.webwizforums.com
Topic: Ratings calculation
Posted By: Rivertree
Subject: Ratings calculation
Date Posted: July 10 2007 at 09:44
 I'm wondering about the calculation result with only one rating. Here's an example where I wrote a review with 3 stars ../album.asp?id=12684 - Forest The calculated result is 3.45 Is it a problem of the algorithm? I might remember that we have a different weighting for collabs and non collabs? Whats's wrong here? ../album.asp?id=12684 - ------------- https://awesomeprog.com/users/Rivertree" rel="nofollow">  |
Replies:
Posted By: avestin
Date Posted: July 10 2007 at 20:31
|
Here's even a better example:
http://www.progarchives.com/album.asp?id=6628 - http://www.progarchives.com/album.asp?id=6628
3.54
(Warning: only 0 ratings) http://www.progarchives.com/album.asp?id=6628#reviews">  So, now the PA server has come to life and has started giving ratings of its own but for some reason doesn't count itself as part of the number of ratings...
Seriously though, there's a systemic (chaos) problem here...
------------- http://hangingsounds.blogspot.com/" rel="nofollow - Hanging Sounds http://www.progarchives.com/ProgRockShopping.asp" rel="nofollow - PA Index of prog music vendors |
Posted By: Rivertree
Date Posted: July 11 2007 at 04:36
|
WOW! It looks like the PA server has its own ratings for the music. So this must be a breakthrough with the development of ARTIFICIAL INTELLIGENCE.  Congratulations! It would be interesting to know who is feeding the server with the songs? BTW - The calculation is raising the results (dynamically??) - yesterday 3.45 - today 3.82 ------------- https://awesomeprog.com/users/Rivertree" rel="nofollow"> |
Posted By: Rocktopus
Date Posted: July 11 2007 at 04:41
The server has changed its mind. Now up to: 3.86 (Warning: only 0 ratings) ../album.asp?id=6628#reviews">  (I'd give it a four myself. Great album) ------------- Over land and under ashes In the sunlight, see - it flashes Find a fly and eat his eye But don't believe in me Don't believe in me Don't believe in me |
avestin wrote:Posted By: Mandrakeroot
Date Posted: July 11 2007 at 08:51
FRANCO MUSSIDA "Racconti Della Tenda Rossa" ratings distribution
3.55 / 5
(1 ratings)
Strange case because the only review is the mine review that have 4.00 and not 3,55 valutation!!! ------------- 
|
Essential: a masterpiece of progressive music (0%)
Good, but non-essential (0%)
Collectors/fans only (0%)
Poor. Only for completionists (0%)
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 09:19
../artist.asp?id=1171 - APHRODITE'S CHILDThe Art Of Démis Roussos And Aphrodite's Child(Boxset/Compilation, 1993)3.45 / 5 (1 ratings) Another case of false rating!!! Because with one rating is impossible 3.45. Also because my rating is 3!!!
-------------
|
Posted By: M@X
Date Posted: July 11 2007 at 09:40
|
Ok -- ill check it out
------------- Prog On ! |
Posted By: M@X
Date Posted: July 11 2007 at 11:11
|
Fixed
------------- Prog On ! |
Posted By: Chris H
Date Posted: July 11 2007 at 11:14
THE BLACK NOODLE PROJECT "And Life Goes On" ratings distribution3.68 / 5
(2 ratings)
Two collabs rated it 4 and 5 stars...
|
Essential: a masterpiece of progressive music (50%)
Posted By: bhikkhu
Date Posted: July 11 2007 at 12:17
|
There is still something going on. My lone 4-star review of Amos Key, "First Key," is calculated at 3.55.
http://www.progarchives.com/album.asp?id=10590 - http://www.progarchives.com/album.asp?id=10590 ------------- a.k.a. H.T. http://riekels.wordpress.com" rel="nofollow - http://riekels.wordpress.com |
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 12:50
Originally posted by M@X
I may have fix this already ... Wich album ? ---------------------------------------------------- How already written from other, the album are much. They would have been lost it lose days to list the all. I am noting that they are all those with 1 rating. For the other I would not know to say. -------------
|
Posted By: Easy Livin
Date Posted: July 11 2007 at 12:58
|
Could you say that again Mandie, I'm afraid I can't understand it. Sorry. |
Posted By: bhikkhu
Date Posted: July 11 2007 at 13:02
|
there is something wrong with the rating system. It's obvious when you look at some albums with only one rating. For example, I am the only one to rate Amos Key, and I gave it four stars. However, the rating is 3.55. ------------- a.k.a. H.T. http://riekels.wordpress.com" rel="nofollow - http://riekels.wordpress.com |
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 13:26
|
I saw that all of the album with 1 rating have this problem: (example) 3,44/ 5 (1 rating) But this isn't possible. In fact If I appraised the album with 4 stars the middle vow wouldn't have to be 4/5? -------------
|
Posted By: Easy Livin
Date Posted: July 11 2007 at 13:34
I think, and I'm speaking from shaky ground here , that the rating shown should not be seen as a straight average of the ratings posted.
It is a figure calculated by taking the rating or ratings of the album, running them through an algorithm which takes other factors into account, an coming up with a result. This result allows the album to find its relative position among all the others.
I know it looks strange that the figure shown for one album is not the same as the only rating for that album, but it reflects all the other constituent parts of the algorithm too.
|
Posted By: Andrea Cortese
Date Posted: July 11 2007 at 13:34
 Prog reviewers has been fired!!!
The site does not need them anymore!
|
Posted By: Andrea Cortese
Date Posted: July 11 2007 at 13:40
|
I've seen that many albums of the ISP genre has been (mysteriously) downrated. Hybla Act 1 by Randone, for example, which was 4,30, now is languishing below the excellence status...
No, the rating is evolving...now at 4,03!
 RANDONE "Hybla Act 1" ratings distribution4.03 / 5
(11 ratings)
Excellent addition to any prog music collection (9%)
  Good, but non-essential (9%)
 Collectors/fans only (9%)
 Poor. Only for completionists (0%)
|
Posted By: bhikkhu
Date Posted: July 11 2007 at 13:44
Well then, that means that people have to start reviewing these lonely albums. Let's start a collaborator drive to get some solid ratings for our underprivileged obscurities. You know, something along the lines of, "For just a few few minutes a day, you could get little "Spark of Light" here the attention it deserves." Who's with me? ------------- a.k.a. H.T. http://riekels.wordpress.com" rel="nofollow - http://riekels.wordpress.com |
Posted By: Ivan_Melgar_M
Date Posted: July 11 2007 at 14:17
|
HT wrote:
I believe it’s a common problem when you add too many variables to an algorithm, we had that same problem once in the office, one of the equations was giving flawed results. For example, if an extra factor is added to albums that have many reviews like + 14%, the system will automatically subtract the same percentage from albums that have a very short number of reviews to keep the equivalence. Take for instance X being the rating for Close to the Edge X = 4 14% the bonus for albums with more than 20 reviews The result will be 4.56 In this case being that there are too many reviews, we would never notice the difference, much less if there are other factors like giving more weight to Collaborators reviews that make it hard to discover. But the computer will automatically subtract that same percentage from albums that have very few reviews. like Y being a review of Amos Key The first Key. Y = 4 -14% will be subtracted from the review being that this album has less than 20 reviews. The new result will be: 3.44 If you add other distorting but necessary factors like less weight for ratings without reviews and extra value for Collaborators reviews, you may get a different final result in the average of only one review. Sounds illogical, but it’s absolutely normal, maybe adding a new variable that will exclude any case in which the reviews are less than two (<2)....Would make it easier to accept. Iván ------------- 
|
Posted By: Rivertree
Date Posted: July 11 2007 at 15:05
CASSIBER "Beauty and the Beast" ratings distribution
3.50 / 5
(0 ratings)
now actually we have a surprising 3.5 calculation for CASSIBER ... ../album.asp?id=6628#postAReview - ------------- https://awesomeprog.com/users/Rivertree" rel="nofollow"> |
Posted By: Rivertree
Date Posted: July 11 2007 at 15:33
I'm quite sure the calculation was OK in the past. So there must have been changes during the last days which have forced this dilemma. Maybe there is a chance to revert to a former state ... ------------- https://awesomeprog.com/users/Rivertree" rel="nofollow"> |
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 16:07
Hey... "2001 Space Odissey" computer is here!!! That sense has give evaluations to the album if then... Already if then a 4 becomes a 2,98 (I wrote a number at random)!!! Then I give to all 1 star and we end it here!!! -------------
|
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 16:09
|
Unfortunately change my manner of to appraise the album!!!
-------------
|
Posted By: Easy Livin
Date Posted: July 11 2007 at 16:15
| There has been a fundamental change to the algorithm used. See the thread in the Prog Music lounge. |
Posted By: avestin
Date Posted: July 11 2007 at 16:36
Bob, does this change apply to albums that have not been rated as well?
See this case for instance - http://www.progarchives.com/album.asp?id=6628 - http://www.progarchives.com/album.asp?id=6628
No ratings done by anyone it says and yet there's an average rating for this album.
------------- http://hangingsounds.blogspot.com/" rel="nofollow - Hanging Sounds http://www.progarchives.com/ProgRockShopping.asp" rel="nofollow - PA Index of prog music vendors |
Posted By: Rivertree
Date Posted: July 11 2007 at 16:55
I can't find any thread in the Prog Music Lounge dealing with this problem  ------------- https://awesomeprog.com/users/Rivertree" rel="nofollow"> |
Posted By: Dean
Date Posted: July 11 2007 at 17:16
the Thread is http://www.progarchives.com/forum/forum_posts.asp?TID=39787&PN=9 - What happened to TOP 100???
However, that thread deals with the relative positions of the album in the Top 100, not how the weighted ratings are calculated.
Out of (now gnawing) curiosity, I would like to see the algorithm that is used to calculate each rating. ------------- What? |
Posted By: Easy Livin
Date Posted: July 11 2007 at 17:25
| mailto:M@x - M@x provided a link somewhere (I thought it was in the PML thread) with a link to details of the algorithm. Don't know about the album with no reviews off hand. Logic dictates it should not have any sort of calculated rating. |
Posted By: Logan
Date Posted: July 11 2007 at 17:34
|
It's more than one album. 1309 albums in the top 4000 all-genres alone by my count last night. As I wrote in that other thread: I wonder why do all the unrated albums in the top 4000 now have a value of 3.86 (by my fatigued checking)? I have a theory, but... I don't find that useful. Taken from the top 4000 most popular albums ../top-prog-albums.asp?year=&mostpopular=4000 - (CLICK) : Every non-rated album in the top 4000 is given a rating of 3.86 and they all fall between 1224 and 2533 (they are surrounded by reviewed albums). Between 2882 and 3286 all the albums have 2 ratings and are given 3.83. Between 2881 and 3153, the first albums only rated once are found, and are given 3.84. And at 3379 to 4000 are all the other albums given one star - at 3.83. All of the albums only rated three times are given a score of 3.85, 3.84, or 3.83. The album at 4000 with one review/rating of 4 by Sean Trane which gets a list rating of 3.83. I see looking at the Kevin Ayers page that the album ratings do vary slightly between the albums only rated once... from 3.79 to 3.83.... ------------- https://www.youtube.com/playlist?list=PLXcp9fYc6K4IKuxIZkenfvukL_Y8VBqzK" rel="nofollow - Duos for fave acts |
Posted By: Dean
Date Posted: July 11 2007 at 17:41
mailto:M@xs - M@x's link pointed to a Wikipedia page on the general principle of weighted averages and another link later on pointed to the comment at the top of the Top 100 page:
Here is some details about how we calculate the average rating of an album and the rank of an album.
Unfortunately that does not give details of the weighted algorithm itself, nor does it explain the anomalies that mentioned by Logan and others. ------------- What? |
Posted By: MikeEnRegalia
Date Posted: July 11 2007 at 17:53
weighted average does not explain these anomalies ... either there are mistakes in the calculation, or M@x is doing something completely different. Either way, I'd be happy to take a look and maybe help him to find a solution. ------------- https://awesomeprog.com/users/Mike" rel="nofollow">Recently listened to: 
|
Posted By: Dean
Date Posted: July 11 2007 at 18:02
Hopefully these anomalies are only affecting the albums with a low number of ratings/reviews, otherwise the entire Top 100 would be completely skewed - and I think someone might notice that and make a bit of a fuss.  ------------- What? |
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 18:05
If one album have zero ratings... The average is zero. Instead they come appraised also those albums. For me a serious anomaly!!! -------------
|
Posted By: avestin
Date Posted: July 11 2007 at 18:09
Anomaly, yes... or a fine example of Artificial inteligence... Although judging by some of the albums that got a rating, an inteligence with not so good taste in music....
------------- http://hangingsounds.blogspot.com/" rel="nofollow - Hanging Sounds http://www.progarchives.com/ProgRockShopping.asp" rel="nofollow - PA Index of prog music vendors |
Posted By: Mandrakeroot
Date Posted: July 11 2007 at 18:12
Well... Is this the Artificial intelligence of the "2002 Space Odissey"'s computer, maybe?   -------------
|
Posted By: Easy Livin
Date Posted: July 12 2007 at 04:02
Chers Mike, I've passed on your offer to M@x!
 |
Posted By: M@X
Date Posted: July 12 2007 at 07:21
|
A major recalculations was made in the past hours, the Average Rating of a particular album is still the same though. And therefore should have fix the 1 rating issues and others , please get back to me ... Now, the difference is in the TOP 100 list ... Before : Log N order by Now: Weighted Avg, comparing an album to all others album in the site The WR of an album is calculated like this: I weight the average number of votes and avg rating of albums against the average rating of a specific album. Every time a review/rating is posted I recalculate the WR for all the albums. Like this: SELECT @avg_num_votes = AVG(n_ratings) FROM CD SELECT @avg_ratings = AVG(CONVERT(FLOAT,rate)) FROM Rating UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings) and the weighted_ratings is used for the position in the TOP 100. What do you think ? Mike ? Max ------------- Prog On ! |
Posted By: MikeEnRegalia
Date Posted: July 12 2007 at 09:01
|
How about this: SELECT @avg_num_votes = AVG(n_ratings) FROM CD WHERE n_ratings > 0 SELECT @avg_ratings = AVG(avg_ratings) FROM CD WHERE n_ratings > 0 AND avg_ratings IS NOT NULL UPDATE CD SET weighted_ratings = ((@avg_num_votes * @avg_ratings) + (n_ratings * avg_ratings))/(@avg_num_votes+n_ratings) WHERE n_ratings > 0 UPDATE CD set weighted_ratings = NULL WHERE n_ratings IS NULL OR n_ratings <= 0 I think it's important to make sure that empty CD entries (0 ratings) are excluded - the WHERE clauses make sure that this is the case. I also based the calculation of @avg_ratings on the computed average of the album - it should have a better performance since there are fewer rows to look at, and my guess is that that value also includes the reviewer weights which makes it a more accurate basis for the formula. Finally I added a WHERE clause to the UPDATE statement and a second UPDATE because for albums without ratings the calculation is not valid: (N*R+n*r)/(N+n) = N*R/N = R for n=0 and r=0 (in other words: Albums without reviews currently get assigned the average rating of all the reviews as their weighted rating). I hope this is helpful to you, M@x ... let me know if you have further questions! ------------- https://awesomeprog.com/users/Mike" rel="nofollow">Recently listened to:
|
Posted By: progressive
Date Posted: July 13 2007 at 11:02
|
This is f*cked system, cannot work this way,,, and people fixing some random ratings... weighted rating averages can remain, but only if we take the absolute average with it. People can check themselves who many ratings some albums have and whether a Reviewer has rated it. We can't go with the popular system because so many have only few ratings.  v v And where's the function who all is being calculated?? I think there could be many was to calculate things (for example three) but they should be shown clearly. And it would be nice if they were in tables that can be modified,, i mean, that the list could be ordered in many ways, for example by year or rating ------------- 
► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄ |
Posted By: Easy Livin
Date Posted: July 13 2007 at 11:43
| What's the point in being angry about it? |
Posted By: progressive
Date Posted: July 14 2007 at 06:48
|
It makes my life a lot more difficult!! Though, i love and hate everything at the same time... http://www.progarchives.com/album.asp?id=13410 there's 3,77 for example, with two 5-stars. Maybe we should vote about it. Why are rating averages changed?? -------------
► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄ |
Posted By: MikeEnRegalia
Date Posted: July 14 2007 at 06:53
|
^ So you think that with two 5 star ratings the album should have a "score" of 5.0 and be listed above long standing albums with 4.x averages? The weighted average simply means that new albums start with the combined average of all the albums in the database and then slowly work their way up. I like it, but I agree that on the website there should be an explanation of how the algorithm works, so that people don't think that it's simply broken. ------------- https://awesomeprog.com/users/Mike" rel="nofollow">Recently listened to:
|
Posted By: progressive
Date Posted: July 15 2007 at 07:40
Yes, and no. It isn't listed above them, at least if there's tow ways to list them and users could choice the way. And the previous system worked well. And there's always top 100 (or 500) lists. There are lists for genres also. Why not for artists? It is only calculating, not taking too much resources. It is very odd that you must look into each album's ratings trying to calculate yourself the absolute average. And who says reviewers' ratings are better? Or that the album is better when there's more ratings. I think proggers are quite intelligent people and because there's so few ratings in many albums, we should take them seriously and not "banning" non-reviewers or single voices. And whatabout situation when there's album with three 4-star votes, one with two five-star votes? And Im still not sure if it is a weighting factor whether you are reviewer or not.... obscure. Very obscure things. Can't go on like this. Dying. -------------
► rateyourmusic.com/~Fastro 2672 ratings ▲ last.fm/user/Fastro 5556 artists ▲ www.progarchives.com/Collaborators.asp?id=4933 266◄ |
Posted By: MikeEnRegalia
Date Posted: July 15 2007 at 09:36
------------- https://awesomeprog.com/users/Mike" rel="nofollow">Recently listened to:
|
Come on, get a life!Posted By: XPEHOPE3KA
Date Posted: July 15 2007 at 11:08
That's the key point! Leave the algorithm for ratings as it was (it took into account who gave the review and how "long" it was). But when you need to list the albums anywhere... It is this place where to use an algorithm which takes the number of reviews into account as well. So the ratings would remain in a state people got used to already (when a collab rate 5, and a noncollab rate 4 the average used to be equal to 4.60 or near - I guess, it was Ok for most visitors), and least reviewed albums won't pop up on tops of rating-lists or something. What's done now is going to make most popular albums even more popular and the least popular albums would receive even less attention. Visitors will just look at the rating (which is now modified by a number-of-ratings-dependent algorithm, but how a visitor would now THAT??) and see it is 3.89 or so - would they be willing to get it?? It doesn't attract even to read the reviews! Hope, you got my point of view: the current algorithm scheme is not that evident to an average visitor (it just isn't natural) and surely would make some potentially good albums unattractive. |
Posted By: Wutu Banale
Date Posted: July 17 2007 at 15:33
This new system isn't good at all. The average for the albums are all screwed up and all. And the new thingy made TaaB go number 1. A spot it doesn't deserve  ------------- Käsittämättömän käsittämätöntä Suurta ja arvostettavaa. |