Quality Control for Reviews

As many people have observed, the quality of reviews on this site has deteriorated somewhat in recent times, which has detracted from the overall quality of the site.

Here are a couple of possible solutions to this problem:

1. Prevent users from submitting reviews or ratings until they have been registered for some period of time, such as a few days or a week. This would prevent "one-off" reviews of very little substance such as those that have appeared for a number of recently released albums by ensuring that the person has an ongoing interest in visiting the site.

2. Introduce a rating system for reviews, so users can rate the quality of any review. A review receiving a high number of "poor" ratings could then be reviewed by a moderator and removed if it is considered to be unhelpful or inaccurate.

This site is a fantastic resource for exploring new music, and it would be good to see some of the "unhelpful" reviews eliminated from the site to further improve it's usefulness.

Mr_Upside_Down wrote: As many people have observed, the quality of reviews on this site has deteriorated somewhat in recent times, which has detracted from the overall quality of the site. Here are a couple of possible solutions to this problem: 1. Prevent users from submitting reviews or ratings until they have been registered for some period of time, such as a few days or a week. This would prevent "one-off" reviews of very little substance such as those that have appeared for a number of recently released albums by ensuring that the person has an ongoing interest in visiting the site.

i thought of that for awhile. like waiting until someone is a senior member in the forum or something like that, but then decided it had little bearing on quality of reviews.

I believe that if you write, say 2 reviews that are horrible and cause people to be pissed at you, then you should be penalized somehow. Though I wouldn't want it to be harsh. Maybe you're bocked from reviewing for a week or blocked from posting in the forum for a couple of days.

I don't know. Besides, I think most of the reviewers here do a fine job. Some are so good that they get well deserved recognition in the forum. I think a month back there was a thread on how amazin Gatot's reviews were.

Your ideas are good, however it would be nice to at least have the mods read the reviews (once again) at least every once in a while and get rid of the reviews that couldn't possibly be helpful, or contain no substance.

Well, manni stated in a post that the Octavarium reviews were being cleansed, if that's a good word to use. I hope that such tactics will be used in all reviews.

Retrovertigo wrote: Your ideas are good, however it would be nice to at least have the mods read the reviews (once again) at least every once in a while and get rid of the reviews that couldn't possibly be helpful, or contain no substance.

The good thing about a rating system would be that the mods would only need to re-read those reviews which received a number of poor ratings from other users. This way, everybody could help the mods determine which reviews need to be reconsidered and possibly removed.

 

Mr_Upside_Down wrote: stonebeard wrote: Mr_Upside_Down wrote: 1. Prevent users from submitting reviews or ratings until they have been registered for some period of time, such as a few days or a week. This would prevent "one-off" reviews of very little substance such as those that have appeared for a number of recently released albums by ensuring that the person has an ongoing interest in visiting the site. i thought of that for awhile. like waiting until someone is a senior member in the forum or something like that, but then decided it had little bearing on quality of reviews. I believe that if you write, say 2 reviews that are horrible and cause people to be pissed at you, then you should be penalized somehow. Though I wouldn't want it to be harsh. Maybe you're bocked from reviewing for a week or blocked from posting in the forum for a couple of days. I don't know. Besides, I think most of the reviewers here do a fine job. Some are so good that they get well deserved recognition in the forum. I think a month back there was a thread on how amazin Gatot's reviews were. I'm not really suggesting that time has a direct bearing on the quality of the review - but if a person was prevented from posting a review until two or three days after registering, it would eliminate the "knee-jerk" reaction of people visiting the site, reading a review they disagree with, posting their own opinions and then leaving, never to return. I think a return visitor is much more likely to be interested in maintaining the quality of the site rather than just sounding-off. I think the review quality rating (point 2) would be more likely to achieve the desired result - but this suggestion would be easier to implement.

possibly, but i believe that most prog fans, being intelligent enough to actually like prog, would recognize that such restraint in reviewing albums is needed and should be used. but since we are discussing this topic now, i know that all prog fans cannot be trusted with that responsibility.

I don't think it's necessary to wait until senior membership to post reviews (not only saying that because I am a newbie) but I do feel that many of the reviews repeat the same thing over and over. It would be a lot of work to have someone approve reviews but there should at least be a word limit. I mean, some of these reviews are around 20-30 words or so repeating everything we've already heard.

tuxon wrote: For the reviewers, when you get high ratings you know you are on the right track with your review, and you will be more motivated for future reviews, and with low ratings you may decide to change your way of reviewing, of course a downside of that is you might get demotivated, which shouldn't be the purpose of such a rating system, so caution is in place (maybe let only moderators/collaborators and perhaps long term members do the rating of a reviewer)

I think the rating system should be open to everybody, rather than just mods/long-term members. After all, the mods can always ignore low ratings if they decide they are unfair. The purpose of review ratings would be to highlight the reviews that need to be looked at by the mods - they would still have the final say as to what should be removed.

As far as the "downside" of demotivating reviewers who get consistently poor ratings is concerned - surely this is a good thing? After all, do we really want to encourage poor reviewers to write *more* reviews?

Dick Heath wrote: Man Overboard wrote: I'd like to add: 3.  Lock an album from review (except to collaborators) until 1-2 months after its release date to help ensure accurate reviews. Good reasoning in the idea. I would add it would also help certain individuals' post-orgasmic response to the receipt and first hearing of thier favourite band's latest platter die way, and perhaps more rational considerations to be made with time. HOWEVER, take an album I put up yesterday, Morgan Fisher's Miniatures - unless a review(s) is provided of the more obscure releases, how is somebody going gauge what the album is about, then with the limited possibility of going after it to buy it? Lock out the abusers and we lock out the genuine fans with something to say? A dilemma.

Is this really necessary though? After all, even the collaborators may get it wrong (according to other fans of a band)... and if somebody can write a good review with a different opinion, surely their opinion deserves to be heard? Really, the only proven way to sort the wheat from the chaff is by the democratic process of voting. If people find a review helpful they will vote for it, and if they find it unhelpful they will vote against it. Reviews that receive a certain number of poor ratings from different sources could be automatically removed or filtered out (or alternatively, reviewed by the moderators). If a review is really poor or inaccurate, this will happen very shortly after the review is posted - whether it be as soon as the album is released, or some months after.

tuxon wrote: Of course some people are born brilliant, others need practice and encouragement

Ok, I understand this was meant as a joke... but really, you don't need to be brilliant to write a decent review. After all, we're not asking for high-calibre journalism here - just an informed opinion based on a thorough appreciation of the album, rather than a useless rant.

Some reviewers will always be better than others, but the aim is really just to exclude the reviews that qualify as nothing much more than a waste of space, not to discourage those who are genuinely trying to contribute.