Top list creating bias? - Progressive Rock Music Forum

Author

Message

Topic Search

Topic Options

Slartibartfast View Drop Down

Collaborator

Honorary Collaborator / In Memoriam

Joined: April 29 2006
Location: Atlantais
Status: Offline
Points: 29630

Topic: Top list creating bias?
Posted: July 23 2008 at 23:28

It's actually a little fun the results that the various algorithms (if I am using the right term) produce. This one was once listed as one of the top albums of 1966:
http://www.progarchives.com/Review.asp?id=135711
It seems that there is a certain floor or ceiling criteria or criterion that albums have to pass to make the lists now.

Released date are often when it it impacted you but recorded dates are when it really happened...

T.Rox

Special Collaborator

Honorary Collaborator

Joined: July 06 2004
Location: Australia
Status: Offline
Points: 9455

Posted: July 23 2008 at 10:18

SNAP!

http://www.progarchives.com/forum/forum_posts.asp?TID=48496

"Without prog, life would be a mistake."

...with apologies to Friedrich Nietzsche

Desoc

Forum Senior Member

Joined: December 12 2006
Location: Oslo, Norway
Status: Offline
Points: 216

Posted: July 20 2008 at 12:24

Ah, I remember too - I think the option to filter by reviews and ratings is fine, but I think the standard filter is excluding too many from the "official" list, that's it.

micky

Special Collaborator

Honorary Collaborator

Joined: October 02 2005
Location: .
Status: Offline
Points: 46838

Posted: July 20 2008 at 12:13

I remember some time back.... M@X screwed around with the parameters of the top lists.. and has tinkered with them... I think minimum reviews became a part of that

The Pedro and Micky Experience - When one no longer requires psychotropics to trip

Desoc

Forum Senior Member

Joined: December 12 2006
Location: Oslo, Norway
Status: Offline
Points: 216

Posted: July 20 2008 at 12:11

? "Monkeying around"?

micky

Special Collaborator

Honorary Collaborator

Joined: October 02 2005
Location: .
Status: Offline
Points: 46838

Posted: July 20 2008 at 12:09

don't agree with the original monkeying around.... but at the same time can understand the logic behind it... if only 5.. or 15 people bother to review it... how can it be considered a 'top' release. The prog world is full of great albums that few have heard...

The Pedro and Micky Experience - When one no longer requires psychotropics to trip

Desoc

Forum Senior Member

Joined: December 12 2006
Location: Oslo, Norway
Status: Offline
Points: 216

Posted: July 20 2008 at 12:05

Isn't it peculiar that only 6 albums qualify for the 2008 top list, and we're soon writing August?

The equivalent 2007 list counts 45 albums. There are three explanations:

1) 2007 was a better year. I doubt it.

2) More albums will qualify in the coming months. Yes, very probable, but that many? Such a steep curve?

3) The threshold to qualify for the "official" list, 25 reviews and rating above 3.5, is too high. Too few albums enter the list, and those that do, are subject to massive review thrashing by those who do not think it deserves a place in the top. If this threshold was lower, more albums would be included, and people would distribute their antipathies more evenly.

Another point against the high threshold is that it really is a critique against the top list algorithm. If albums with fewer than 25 ratings, or at least 10 ratings, doesn't deserve a place in the top 10 in July, then the algorithm is too biased in favor of albums with few ratings...

Suggestion for improvement:

1. Make "no reviews" and "no rating" the standard for the official list

2. Continue to allow people to make their own standards for thresholds, and allow computers to remember the settings

3. Construct an understandable algorithm

Author	Message Topic Search Topic Options Post Reply Create New Topic Printable Version Translate Topic
Slartibartfast Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Collaborator Honorary Collaborator / In Memoriam Joined: April 29 2006 Location: Atlantais Status: Offline Points: 29630	Topic: Top list creating bias? Posted: July 23 2008 at 23:28
	It's actually a little fun the results that the various algorithms (if I am using the right term) produce. This one was once listed as one of the top albums of 1966: http://www.progarchives.com/Review.asp?id=135711 It seems that there is a certain floor or ceiling criteria or criterion that albums have to pass to make the lists now.
	Released date are often when it it impacted you but recorded dates are when it really happened...

T.Rox Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Special Collaborator Honorary Collaborator Joined: July 06 2004 Location: Australia Status: Offline Points: 9455	Posted: July 23 2008 at 10:18
	SNAP! http://www.progarchives.com/forum/forum_posts.asp?TID=48496
	"Without prog, life would be a mistake." ...with apologies to Friedrich Nietzsche

Desoc Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Forum Senior Member Joined: December 12 2006 Location: Oslo, Norway Status: Offline Points: 216	Posted: July 20 2008 at 12:24
	Ah, I remember too - I think the option to filter by reviews and ratings is fine, but I think the standard filter is excluding too many from the "official" list, that's it.

micky Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Special Collaborator Honorary Collaborator Joined: October 02 2005 Location: . Status: Offline Points: 46838	Posted: July 20 2008 at 12:13
	I remember some time back.... M@X screwed around with the parameters of the top lists.. and has tinkered with them... I think minimum reviews became a part of that
	The Pedro and Micky Experience - When one no longer requires psychotropics to trip

Desoc Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Forum Senior Member Joined: December 12 2006 Location: Oslo, Norway Status: Offline Points: 216	Posted: July 20 2008 at 12:11
	? "Monkeying around"?

micky Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Special Collaborator Honorary Collaborator Joined: October 02 2005 Location: . Status: Offline Points: 46838	Posted: July 20 2008 at 12:09
	don't agree with the original monkeying around.... but at the same time can understand the logic behind it... if only 5.. or 15 people bother to review it... how can it be considered a 'top' release. The prog world is full of great albums that few have heard...
	The Pedro and Micky Experience - When one no longer requires psychotropics to trip

Desoc Members Profile Send Private Message Find Members Posts Add to Buddy List Reviewer page Forum Senior Member Joined: December 12 2006 Location: Oslo, Norway Status: Offline Points: 216	Posted: July 20 2008 at 12:05
	Isn't it peculiar that only 6 albums qualify for the 2008 top list, and we're soon writing August? The equivalent 2007 list counts 45 albums. There are three explanations: 1) 2007 was a better year. I doubt it. 2) More albums will qualify in the coming months. Yes, very probable, but that many? Such a steep curve? 3) The threshold to qualify for the "official" list, 25 reviews and rating above 3.5, is too high. Too few albums enter the list, and those that do, are subject to massive review thrashing by those who do not think it deserves a place in the top. If this threshold was lower, more albums would be included, and people would distribute their antipathies more evenly. Another point against the high threshold is that it really is a critique against the top list algorithm. If albums with fewer than 25 ratings, or at least 10 ratings, doesn't deserve a place in the top 10 in July, then the algorithm is too biased in favor of albums with few ratings... Suggestion for improvement: 1. Make "no reviews" and "no rating" the standard for the official list 2. Continue to allow people to make their own standards for thresholds, and allow computers to remember the settings 3. Construct an understandable algorithm