When rating other people's translations I am rewarded with points when agreeing with a majority opinion, even when this is clearly erroneous. Is there a mechanism awarding points ex post in case the majority opinion follows my choice?
I think rewarding incorrect answers that "go with the crowd" slows down the process by which the crowd converges to the right answer. Some people then might knowingly rate a wrong answer as a correct answer, believing this will bring them points. A solution would be a large reward for the "courageous" minority that voted for what later turned out to be the correct answer. By this I mean: once the majority opinion flips and follows an opinion of a minority the original minority receives twice the amount of points, for example. Of course, multiple flips would still be a problem, so a more complicated criterion could be used once the sentence is closed as translated and the final correct solution is known. Obviously, the mechanisms for immediate reward should be preserved.
Couldn't there just be a reward at first, say a point, and then later if you correctly voted and didn't vote randomly then you would get more; however, if you voted up an erroneous solution you would lose the point gained when the translation is final. Then random voting would be discouraged, but voting in general would still have incentive and if you vote correctly then you are rewarded more, thus encouraging it.
I think it is premature to try to fine-tune the rewards process, when the meanings of the translation evaluations are so vague. I have said for a long time that I think there should be at least four levels. With the present three levels, I often find it hard to choose which one I think is appropriate, and I'm sure other people are using standards which differ greatly from mine. Here are a few problematic questions: (1) Is a translation "perfect" if it conveys the idea of the original sentence reasonably clearly, but contains one or more unnatural turns of phrase, or is idiomatically awkward in other ways? How does one distinguish a really excellent translation if such translations are also rated "perfect"? (2) If there is only one "error" in a translation, but it completely changes the meaning of the sentence so that the original meaning is lost, is that "some mistakes" or "many mistakes"? The description "many mistakes" doesn't seem to fit, since there may be only one word wrong, but it is a much worse translation than one which conveys the meaning correctly despite poor choices of several prepositions, for example. I recommend at least four levels, and considerably more guidance in how to rate sentences, so that at least we are all trying to assign the same meanings to them.
I have thought the same. I think this is a side effect of the "crowd-sourcing" principle... in that you only know the correct answer when all the crowd has voted, so early ratings are error prone, so rewards are meaningless early on. I would agree that rewards should only be given at the end, but then again that would not encourage people to answer for rewards now. So I guess this is a fundamental dilemma in this application.
When does a post close? After a certain number of votes? Most seem to close quite quickly so not receiving points until closing shouldn't be too problematic. We all learn by seeing the BEST translation, which may not be the most popular. Often after seeing several translations I have a better understanding of what it is or how best to word it.
I also found the given definitions for each of the three levels problematic as explained by fuonk. I ended up using the icon as a more obvious guide and finding my own face looking unhappy with only one clear mistake. As to the number of levels, I'd favor the present, simple, three-level system, with improved definitions. (Getting sidetracked from polto's original point...?)