5 stars vs. 4 stars

I was reading maani's review of Neal Morse - Testimony:

http://www.progarchives.com/Review.asp?id=43607

First of all - I don't want to criticize this review in any way. It just made me think about the rating system again:

There is very little I can add to all the wonderfully written and spot-on 5-star reviews - except for the fact that this is not a 5-star album. Not to say it isn't great - it is. But on the same level as In The Court, Selling England, Close to the Edge, Thick as a Brick and Dark Side? I think not.

So what he's saying is essentially that the 5 star rating is reserved for only a handful of albums. That's a big problem for me, because people who think this way are a minority on this website, judging from the huge amount of reviews and ratings that are already in the database. I understand the reasons behind this approach, but it wasn't accepted by the public. It has failed, and that problem cannot be rectified without deleting all reviews and start from scratch, which is out of the question.

If you look at the statistics, it's obvious that 5 star ratings are commonplace in the archives, they're not as special as they are described to be in the rating instructions. Many albums have average ratings well above 4.0, which is only possible if MANY people gave 5 star ratings.

So when rating an album which you consider to be very very good, an exceptionally well made album, but not as good as CttE or SEbtP ... what should you do? There are several approaches:

• Rate the album 4 stars, arguing that 5 stars are reserved for a dozen albums from the 70s.
  This approach punishes the album in terms of the average rating, because most people use 5 star ratings for really good albums, and 4 stars mean "just good". Think about it: Even if many more people gave Neal Morse - Testimony 5 stars, the album wouldn't get in any position to compete with CttE or SEbtP in the charts.
• Rate the album 5 stars.
  This is better IMO. This won't put it on the same level as CttE or SEbtP, the average rating and number of reviews will put the albums in proper relation.
• Rate the album 5 stars, but explain in the review that it's not as good as the masterpieces of the 70s, but a real gem nonetheless (which every fan of that genre - and related genres - should have).

I think the real problem is that there are only two rating options for really good albums, 4 stars and 5 stars. I have about 300 prog CDs, and of those easily 30-40 prog albums are candidates for 4.5 stars. They're not as good as CttE or SEbtP, but they're too good to be put on the same level as the maybe 150-200 prog albums in my collection which I would rate 4 stars.

The bottom line: I will use 5 stars more lenient in my reviews, but explain in the review why I don't want to give 4 stars.

 

Edited by MikeEnRegalia

BaldFriede:

That wasn't my point. You should substitute those albums with your absolute favorite albums, just for the sake of this discussion.

BaldFriede wrote: Your argument starts with a premise not all might agree with. Are the albums you listed (CTTE, SEBTP, TAAB and DSOTM) really 5 star albums? Some may agree to that, but I for one have my doubs about at least three of these albums, maybe all 4 even; anyway, I know a LOT of albums that I honestly think are a lot better than the ones named. Unfortunately, not many may be of the same opinion as I about these albums that I value so high. But what I want to say is this: Every album should be judged on its own and not in comparison with others. An album may be so unique it can't be compared to anything else. The explanation of the five-star classification system gives you enough clues how you should proceed when rating an album.

And exactly these clues are wrong. Read my post again, I explain why when following these guidelines to the letter, you might actually even punish your album and make it appear to be less good than it actually is (for you).

This punishment talk is just bullsh*t imo. Does it mean that the ones who review strictly (and how it should be) should fall to the level of the ones who give 5 stars easily? No thanks, I would rather stop reviewing than fall to that level. I have noticed that the ones who review strictly are the ones who clearly have the best knowledge. If you haven't heard a lot of albums it is hard to do the rating correctly when you don't have enough where to compare. This is a problem that cannot be solved because the reviews are open to everyone.

This should be easy to understand but people just don't get it:

Essential: a masterpiece of progressive music (0%)

Excellent addition to any prog music collection (0%)

Good, but non-essential (0%)

Collectors/fans only (0%)

Poor. Only for completionists (0%)

For me, four stars never means "just good". The ones I rate with four stars are often among my very favourites.

geezer wrote: This punishment talk is just bullsh*t imo. Does it mean that the ones who review strictly (and how it should be) should fall to the level of the ones who give 5 stars easily? No thanks, I would rather stop reviewing than fall to that level. I have noticed that the ones who review strictly are the ones who clearly have the best knowledge. If you haven't heard a lot of albums it is hard to do the rating correctly when you don't have enough where to compare. This is a problem that cannot be solved because the reviews are open to everyone. This should be easy to understand but people just don't get it: ... For me, four stars never means "just good". The ones I rate with four stars are often among my very favourites.

I'm not saying that we should resort to giving only 5 or 1 star ratings. I'm just saying that we can be a little more lenient. There's no exact way to determine the rating anyway. I don't intend to limit my 5 star ratings to just 5 albums. I don't want to review one of my absolute favorite albums, give it 4 stars and say "sorry folks, I cannot give it 5 stars because XYZ is better".

I'll try to maintain a 1:4 ratio between 5 and 4 stars. But even that leaves me with 30-40 5 star albums. And that's just 10% of my collection.

cobb wrote: Mike, it is where the rating system fails. If all data input is not governed by the same set of strict guidelines then the database is biased. Any information returned from this will, of course, include this bias. Personally, I think the guidelines of essential.... etc are too difficult to adhere to. Essential compared to what? A masterpiece of progressive music. Only time can tell this. How many of us can recognise a masterpiece of progressive music? Certainly not me. I know whether I think anything new (as in new release) I hear (after x amount of listens, to give it time to breed and grow) is a great piece of music, but I cannot tell whether it will become a masterpiece. I don't know whether I am placing too much into masterpiece- to me it means that a work will stand the test of time and be accessible to later generations.

Exactly. Only the combined results from all ratings can give an indication which albums might be THE masterpieces of prog rock. And even then, they're just the most popular masterpieces, as people like BaldFriede/BaldJean and a lot of other people (like myself) would say. Peter Hammill's Solo albums might be better than anything anyone has ever done - but only for some people. And Devin Towsend - Terria might also be the best music EVER - for SOME people.

I think that as long as you're not rating every album you like 5 stars - it's ok. It also depends on whether you just review your favorite albums, or also many albums which you don't like. IMO there are some reasons when a 5 star rating isn't appropriate:

• You know an album that is MUCH better, and is in the same genre (the albums are comparable).
• You like it very much, but you know that it's not THAT progressive. There are other albums which you don't like as much, but are more progressive

 

When I rate albums, I choose the rating on this similar way.

*****  -  A complete masterpiece that everyone should own. The music is extremely enjoyable.

**** - A highly enjoyable album which may have a few flaws, but it is still highly recommended to anyone.

*** - A good but not great album that should be left for later when buying CDs

** - A mediocre progressive album that does not offer much excitement and interest.

* - A very bad album that should never be bought unless it is for collection. The music is not interesting and/or is unlistenable.

With you MER

I noted yesterday that the majority of reviews provided by specifically collaborators were hitting 4 or more often 5 stars - somebody went berserk with a run of Procol Harum albums rated 3 or 5...................  Okay favourites will get posted more than the unloved/unplayed. But somewhere between the rules of the local star system and the reviewers, a masterpiece is not  a rarity and rather a commonality. 

1. There any many albums listed as 5 starshere - i.e. 'should be in every prog fan's collection' - which I won't touch with a bargepole. So one criteria a reviewer has ask him or herself: would everyone want this album in their collection?

2. Until  people have hear a large number of prog albums, coming of every type of sub-genre, then I would argue that they are not particular well equipped to judge a masterpiece. There can be considerable differences between a personal favourite album and a masterpiece.

3. There is that element of 'standing the test of time', as one measure of quality, which new recordings just can't meet. Is Francis The Mule really a masterpiece, how will anybody judge it in 2 or 10 years time?

4. Can any one band produce more than one masterpiece - almost a contradiction in terms?

maani wrote: As for Mike's statement that "Even if many more people gave Neal Morse - Testimony 5 stars, the album wouldn't get in any position to compete with CttE or SEbtP in the charts," that is simply false.  Indeed, this was proven false during the Octavarium debacle, when so many people joined the site just to give it 5 stars that within 48 hours it was in the Top Ten of all time!  Indeed, one cannot argue the above point on the one hand and then state in the same breath that "Rate the album 5 stars...this won't put it on the same level as CttE or SEbtP, the average rating and number of reviews will put the albums in proper relation." (Emphasis mine.)  That sentence completely contradicts the first thought: i.e., if the average rating and number of reviews determines the final rating, then clearly if dozens of people now decided to review the album and give it five stars, it would in fact end up in the same "pantheon" as CTTE and DS.

 I don't think my statement is false. Your example (Octavarium) was the reason for me PMing M@x with a suggestion how to improve the algorithm, and here we are - albums like Octavarium are now nowhere near the top 10, although the avg rating and number of reviews remain the same as before.

Some Prog-Metal albums are currently climbing the ladder - Psychotic Waltz - A Social Grace for example. But as soon as they reach a certain position, some people who don't like Prog-Metal will write some negative reviews and push them down again. It's ok, their opinion is as valid as any other. And if an album should really get more than 80 5 star rating and almost no negative ratings - would that not be a sign of a true masterpiece?

BTW: The current algorithm dampens the effect of the number of reviews, so that for albums with more than 150 ratings the average rating becomes much more important.

maani wrote: All that said, I have always been frustrated by the 5-level rating system, since I do not believe it gives enough leeway for accurate rating.  Indeed, I have been arguing for a half-star system for over two years - but, alas, it is not to be.

I would prefer a half-star rating as well, but it's too late to change anything, without starting from scratch, which is out of the question.

maani wrote: Obviously, no system is perfect, since some people are always going to give an album five stars simply because they love it alot; i.e., they make no effort whatsoever to "put it in perspective" with the whole of prog, much less the history of prog.  However, I believe a half-star rating system would mitigate that at least a little more than the current system does.

There is no person who knows all of prog music. I'm listening to prog 2 hours each day, and I'm nowhere near knowing even half of all I should know. Having said that, why should we demand that everyone tries to be completely objective?

maani wrote: Finally, even given all of the above verbosity (), I do not think that Testimony is deserving of "masterpiece" status.  It is flawed, though perhaps not very much.  Still, if, say, 15%, or even just 10%, of the album does not rise to or maintain the highest possible level of excellence (or is not truly "prog"), then I cannot in good conscience give it 5 stars. Peace.

IMO it might deserve a 5 star rating on this website, but it's not a masterpiece. Mathematically speaking, a 4 star rating will decrease its average rating, and a 5 star rating will increase it.

For me it is a typical case of an album between 4 and 5 stars ... "One" is an album that I really see as a masterpiece, and I would rate it 5 stars without any hesitation.