Forum Home Forum Home > Site News, Newbies, Help and Improvements > Help us improve the site
  New Posts New Posts RSS Feed - Question About Rating System
  FAQ FAQ  Forum Search   Events   Register Register  Login Login

Topic ClosedQuestion About Rating System
 Post Reply Post Reply
Author
Message
  Topic Search Topic Search  Topic Options Topic Options
AmericanProgster View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 28 2008
Location: Manassas, VA
Status: Offline
Points: 228 		Direct Link To This Post Topic: Question About Rating System
    Posted: February 13 2009 at 17:22
Don't know if I put this in the right place, but anyways I'm a bit confused with the rating system. Let's say that an album has an overall rating of 3.50 and 6 votes (making it an excellent addition to any prog collection, etc). 50% gave the album 4 stars, 33% gave it 3 stars. One person comes in and gives it 5 stars and the rating goes down! How does that make sense!?!  Shouldn't it go up!?!

I know that only one out of 5 other people gave it a 5 star rating but how does it cause it to go down???


Edited by AmericanProgster - February 13 2009 at 17:23
https://dawnapproach.bandcamp.com/

https://soundcloud.com/dawn-approach
Back to Top
Pnoom! View Drop Down
Forum Senior Member
Forum Senior Member


Joined: September 02 2006
Location: OH
Status: Offline
Points: 4981 		Direct Link To This Post Posted: February 13 2009 at 17:31
It's a weird algorithm and I don't know exactly how it works, but it's not just avg rating, but also takes into account the number of ratings.

I presume a more knowledgeable member will shortly explain it better.
Back to Top
AmericanProgster View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 28 2008
Location: Manassas, VA
Status: Offline
Points: 228 		Direct Link To This Post Posted: February 13 2009 at 17:35
Then what's the point of giving an album 2 stars if, depending on the circumstances, a 5 star rating will decrease the overall rating just as effectively?

Edited by AmericanProgster - February 13 2009 at 17:36
https://dawnapproach.bandcamp.com/

https://soundcloud.com/dawn-approach
Back to Top
Sckxyss View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: May 05 2007
Location: Canada
Status: Offline
Points: 1319 		Direct Link To This Post Posted: February 14 2009 at 02:00
A five star rating should never decrease the overall rating... I think you must have misinterpreted something that happened. What specific album are you talking about?
Back to Top
Mr ProgFreak View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: November 08 2008
Location: Sweden
Status: Offline
Points: 5195 		Direct Link To This Post Posted: February 14 2009 at 03:19
http://en.wikipedia.org/wiki/Bayesian_average

It's difficult to understand, but the formula can have the effect you describe. It uses two constants: the average album rating across the whole database (let's call it AVG) and the average number of ratings per album (let's call it N). If the number of ratings for any given album is below N and the average of those ratings is below AVG, then adding another rating that's above AVG can actually cause the resulting average to decrease. This is because with this formula the more ratings an album has, the more weight its own average has in relation to AVG.

In a nutshell: For all albums with very few ratings, such strange effects might occur. But since I don't know which values M@x currently uses as AVG and N, I can't do the calculation myself to verify.


Edited by Mr ProgFreak - February 14 2009 at 03:20
Back to Top
AmericanProgster View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 28 2008
Location: Manassas, VA
Status: Offline
Points: 228 		Direct Link To This Post Posted: February 14 2009 at 07:08

It was this album that I gave a 5 star rating to:

http://www.progarchives.com/album.asp?id=6603

I didn't misinterpret anything, it was a 3.50 before I rated it and it was a 3.41 after I rated it.
 
And thanks for the info Mr. ProgFreak, but you'll forgive me if I think thats dumb!TongueLOL
https://dawnapproach.bandcamp.com/

https://soundcloud.com/dawn-approach
Back to Top
Tony R View Drop Down
Special Collaborator
Special Collaborator
Avatar
Honorary Collaborator / Retired Admin

Joined: July 16 2004
Location: UK
Status: Offline
Points: 11979 		Direct Link To This Post Posted: February 14 2009 at 16:20
Well it works and has worked for quite a while now.

\bar{x} = {{C\bar{m} + \sum_{i=1}^n{x_i}} \over {n + C}}

Simple.
Back to Top
The Quiet One View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: January 16 2008
Location: Argentina
Status: Offline
Points: 15745 		Direct Link To This Post Posted: February 14 2009 at 16:22
Originally posted by Tony R Tony R wrote:

Well it works and has worked for quite a while now.

\bar{x} = {{C\bar{m} + \sum_{i=1}^n{x_i}} \over {n + C}}

Simple.


LOL
Clap
Back to Top
The Quiet One View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: January 16 2008
Location: Argentina
Status: Offline
Points: 15745 		Direct Link To This Post Posted: February 14 2009 at 16:26
Originally posted by AmericanProgster AmericanProgster wrote:

Then what's the point of giving an album 2 stars if, depending on the circumstances, a 5 star rating will decrease the overall rating just as effectively?


Not neccessary about the topic, but reffering to this post of yours.

You shouldn't rate albums high or low just so as the album raises or gets down respectively. What counts is your review, and as well as the INDIVIDUAL rating.

Of course I may have misunderstood your post, and you were just reffering to a problem of the site, and in that case, I'm sorry.Embarrassed
Back to Top
AmericanProgster View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: September 28 2008
Location: Manassas, VA
Status: Offline
Points: 228 		Direct Link To This Post Posted: February 14 2009 at 16:57
Oh trust me I don't put reviews just to make the overall rating go up or down, I was just a little confused when it happend. The reason why I might just put a rating, and not an actual typed review, is because I either have a hard time coming up with a good review or I don't have enough words to meet the minimum requirement and they get removed when someone writes more than say 200 words.
 
But hey whatever works for the rating system is fine with me, I was just a little surprised thats all. Thanks for the simple formula!LOL
https://dawnapproach.bandcamp.com/

https://soundcloud.com/dawn-approach
Back to Top
Mr ProgFreak View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: November 08 2008
Location: Sweden
Status: Offline
Points: 5195 		Direct Link To This Post Posted: February 15 2009 at 02:26
Originally posted by cacho cacho wrote:

Originally posted by Tony R Tony R wrote:

Well it works and has worked for quite a while now.

\bar{x} = {{C\bar{m} + \sum_{i=1}^n{x_i}} \over {n + C}}

Simple.


LOL
Clap


Actually it's really simple - considering the normal formula for the average (arithmetic mean) is - visually - already the most complex part:



BTW: For those of you who don't know the notation: It's basically saying "add all the ratings and then divide the result by the number of ratings".


Edited by Mr ProgFreak - February 15 2009 at 02:29
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575 		Direct Link To This Post Posted: February 15 2009 at 07:09
The above explanations have over-complicated the answer a little - the weighted averages method is NOT used to calculate an album average - it is only used to calculate chart position.
 
In this particular instance (RAGNAROK - Ragnarok) there was a two year gap between your rating and the previous one - the rating system has undergone several changes in that time and your rating forced a recalculation based on the current system - the previously displayed result used an old system.
What?
Back to Top
Mr ProgFreak View Drop Down
Forum Senior Member
Forum Senior Member
Avatar

Joined: November 08 2008
Location: Sweden
Status: Offline
Points: 5195 		Direct Link To This Post Posted: February 15 2009 at 08:32
^ if that's true then M@x should try to recalculate all the old values ... when I make a change to the algorithm at PF then that's the first thing I would do.
Back to Top
M@X View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Co-founder, Admin & Webmaster

Joined: January 29 2004
Location: Canada
Status: Offline
Points: 4051 		Direct Link To This Post Posted: March 05 2009 at 11:32
Dean is right, the Weighted Ratings is use to calculate the position in TOP 100 for example.


The AVERAGE RATINGS  is use to give the STAR but it's weighted in a way using the following elements.

Collaborators Rating x10 the value of their rate
Rating with a review from other members X3 the value
Rating only X1 the value


Anyway I don't think it explains why the AVG_RATING drop after a 5star rate.

I think I have a explanation , and this is why it occurs rarely.
( I did not think about that when I built the system).


Since the value of the rating by a COLLABORATOR for a spec. album is X10. When , one member , for example was
not yet a collab. and rate the album X, and give 3 star. The avg. rating of albumX was calculated base on the X3 or X1 law.

If that member become a collab. (or drop from collab.) I do not re-calculate, on that status change the new avg_rating, unless someone
post a new rating on this album. So the next person who give a 5star will force to recalculate the avg_rating, and now using the law that the
old member (now a collab.) his 3stars, worth 10 X 3 stars rating, so the 5 stars, would create the avg_rating to drop.

This is the best explanation for now ...

Let's disuss about this ok ?

Any suggestion
Prog On !
Back to Top
M@X View Drop Down
Forum & Site Admin Group
Forum & Site Admin Group
Avatar
Co-founder, Admin & Webmaster

Joined: January 29 2004
Location: Canada
Status: Offline
Points: 4051 		Direct Link To This Post Posted: March 05 2009 at 11:53
I just check out the album who caused the problem and it's exactly the case I mentionned
Prog On !
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575 		Direct Link To This Post Posted: March 05 2009 at 12:26
Clap Excellent - that's another unexplained anomaly explained Thumbs Up, thanks M@X
What?
Back to Top
crimson87 View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: January 03 2008
Location: Argentina
Status: Offline
Points: 1818 		Direct Link To This Post Posted: May 02 2009 at 20:27
I have calculated the rating of BSS by it's percentages and I got an average of 4.229 instead of 4.15. I did as it follows
 
(Percentage of 5 stars*total number of reviews)*5
          "              "  4    "                       "                           4
          "             "   3                            "                            3
And so on , I added all that and divided it by the total number of reviews
 
I guess is does not take the " Prog reviewers/collaborators effect" into account , which in BSS's case was negative: -0.08
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575 		Direct Link To This Post Posted: May 02 2009 at 20:37
^ You have also got rounding errors in your calculation. The percentages are rounded to the nearest whole percentage.
What?
Back to Top
crimson87 View Drop Down
Prog Reviewer
Prog Reviewer
Avatar

Joined: January 03 2008
Location: Argentina
Status: Offline
Points: 1818 		Direct Link To This Post Posted: May 02 2009 at 20:44

That could be one of the reasons , but from what I have read on previous discussions. The rating is weighted against an average rating of the whole database and the whole number of reviews against the album ones isn't it?

That's quite weird , why would you have to mix "Earthbound" with "Close to the Edge"?
Back to Top
Dean View Drop Down
Special Collaborator
Special Collaborator
Avatar
Retired Admin and Amateur Layabout

Joined: May 13 2007
Location: Europe
Status: Offline
Points: 37575 		Direct Link To This Post Posted: May 02 2009 at 20:48
Originally posted by crimson87 crimson87 wrote:

That could be one of the reasons , but from what I have read on previous discussions. The rating is weighted against an average rating of the whole database and the whole number of reviews against the album ones isn't it?

That's quite weird , why would you have to mix "Earthbound" with "Close to the Edge"?

That method is only used when compiling the Top 100 chart position - so therefore it is natural to mix "Earthbound" with "Close to the Edge" if you are going to put them in the same chart.
 
The actual value for each album does not use the average rating for the whole database, so the 4.15 for BSS is purely on the ratings it received.
What?
Back to Top
 Post Reply Post Reply

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.352 seconds.
Donate monthly and keep PA fast-loading and ad-free forever.