https://www.duolingo.com/jsbender

Does the leaderboard system for translations make sense?

I like having the rankings of who has done the most translations the last day and the last week, but I'm also thinking about ways it could be improved. It doesn't take into account the quality of the translations and maybe there should be another list with the people who have the most well-rated translations. I think the best way to decide whether a translation is good is by how many good ratings other users give it.

6 years ago

11 Comments


https://www.duolingo.com/jsbender

What undesirable effects do you have in mind? I guess friends could always rate each other's translations and rack up points that way. It just worries me a little that the motivation to translate as many sentences as possible would lead to people blowing through them without much thought or care, and there should be some motivation to do it well.

6 years ago

https://www.duolingo.com/amichail

But that might have an undesirable effect on how people rate translations, thus lowering the quality of the translated web pages.

6 years ago

https://www.duolingo.com/ClassicBookworm

@jsbender I think you're right, there should be a balance between points and quality. Perhaps something like the bonus points for lessons. There could be bonus points awarded every time a translation is upvoted.

6 years ago

https://www.duolingo.com/jsbender

Yeah. Getting more points for having a sentence rated well is a good idea. I think the rating system should be simplified to just a thumbs up or thumbs down, or the two arrows like the ones next to these Questions and Insights. If you get a positive vote, you could get bonus points, and if you get negative points it could take away points. That might be a little discouraging to lose points though, but it would encourage people to try harder on their translations.

6 years ago

https://www.duolingo.com/Arjofocolovi
Arjofocolovi
Mod
  • 21
  • 18
  • 17
  • 9
  • 8

@ jsbender I strongly disagree with your idea of thumb up thumb down. It's doesn't fit for translations. Translations are not either "good" or "bad".

There are "ok but needs improvement" there are "doesn't make any sense at all" there are "not used in everyday language"...etc

In fact I think 3 different ratings is fine. This way only really good translations will get points. If you follow the -1 0 +1 rule, that is.

And I think the idea to limit the ratings to only 2 might do the exact opposite to what you want : Let me explain. It's very unlikely that there are any professional translators here. So most of the translations will not be great. Globally I think it will goes like 10% great 60% not bad and 30% bad. And I'm being nice here...

If you have only two ratings, you will have to sort the 60% out. People simply won't rate them bad, because they don't want to be unfair. "If it's not bad, I guess it's OK" Binary choices make stupid results, as you can see on youtube videos for example, where bad videos (and sometimes REALLY bad videos) have a lot of thumbs up...

So to make it short, yes, I agree with you for the idea of improving one's rating on the quality of his translations. But I disagree with a binary rating. It would NOT encourage people to try harder, because people will simply give them points for translations that are not high quality.

6 years ago

https://www.duolingo.com/jsbender

@ Arjofocolovi Very good points. Though you could think of no vote at all as a third, neutral option. For example, vote down a bad sentence, vote up a good sentence, and don't vote for a mediocre one. Calling a sentence mediocre with that flat-smiled face in the middle doesn't give much feedback about it since, as you said, it covers at least %60 of the cases. One reason I had the up and down arrows in mind was that I was thinking that in a list of all the submitted translation, an up vote would move a sentence up the list and a down vote would move it down. That way the best translations would move to the top and be seen first, and also would be used by the computer to compare to first against new submissions. In this system a neutral button would just keep the translation in the same place and not really have an effect. But now I'm talking about ranking the translations and not about giving out points. What are your thoughts?

6 years ago

https://www.duolingo.com/Arjofocolovi
Arjofocolovi
Mod
  • 21
  • 18
  • 17
  • 9
  • 8

Well, I'm thinking of both actually : Giving points and taking points to people, following their translations, and I still think a neutral vote is needed, I'll explain why below. And also ranking translations between them so the most rated are seen first.

To achieve that, if we combine the actual system, giving -1 0 +1, and sorting translations by points, so the first would be on top, we could obtain something quite good, I think.

But, correct me if I'm wrong, I think this kind of ranking is already in place no ? I've always seen translations with more points on top so far.

If it's not the case, well I don't have a better idea than yours : simply sort translations by points, the first up, the last down.

-

But as for the ratings, if we don't make neutral active rating, people will tend to vote anyway. I mean : If I give you a box with two buttons on it one tagged "bad" and the other "good", without saying anything else than "Push whichever you want to rate this box". Any person would think naturally that they "must" make a choice. Only few people will not push any button. So again, we would have poor rankings at the end, because people will vote for something that doesn't fit.

-

Also, without a neutral active vote, people who vote neutral wouldn't be represented, and we won't know how many of them think this way.

-

For example : A translation with 2 bad and 10 Good is one thing. Take the same translation, but this time you have 2 bad, 10 Good and 55 medium ! In both systems the same number of people voted. Only in the second one we know that in majority, people rate this translation as only "average" and not "good" as the other system would pretend.

-

That's why I think an ACTIVE neutral vote is absolutely necessary.

6 years ago

https://www.duolingo.com/jsbender

Yeah I think you have convinced me. I didn't realize if they are already sorting the best to the top. It would be nice to be able to browse the submitted translations without having to submit one of your own, as a way to learn from others and if you're not at a level to translate that article yet. That's when it would be great to have the best translations at the top.

6 years ago

https://www.duolingo.com/ClassicBookworm

@jsbender The previous version of the translation app used to let you see ALL the previous translations and their votes. That was sometimes very useful when the one or two translations you got to vote on weren't very good. I'd love to have that feature back, as well as the option to see the whole article composed of the highest rated translations.

6 years ago

https://www.duolingo.com/Faya
Faya
  • 10
  • 6

A comment on "People simply won't rate them bad, because they don't want to be unfair": I hope that people are not too diplomatic when it comes to rate translations. If a translation is clearly wrong, well, then it is wrong and should be marked as such. I think, nobody should feel offended in that case.

6 years ago

https://www.duolingo.com/Arjofocolovi
Arjofocolovi
Mod
  • 21
  • 18
  • 17
  • 9
  • 8

@ Faya

-

I hope so too, but unfortunately I don't think that's how people will do it. It's not only a matter of how we want to be fair. Combined to the fact that we have only two choices, if we see mistakes in the translations but overall it's OK, we won't rate it bad. And because there would be only one other choice, we will rate it good, even if it's not. Only few people in this case would not rate it at all.

6 years ago
Learn a language in just 5 minutes a day. For free.

Learn a language in just 5 minutes a day. For free.