1. Forum
  2. >
  3. Topic: Duolingo
  4. >
  5. Add a "none of the above" opt…


Add a "none of the above" option to translation rating

Often, none of the three translated sentences I'm asked to vote on are correct so I simply pick the best of the worst. The process of zeroing in on the correct translation could be sped up by allowing a user to select an option that indicates that none are correct. Without this option, doesn't it artificially promote incorrect translations? Maybe I don't have a clear understanding of how the final translation is selected but I know that in most documents I see incorrect translations marked 100% and listed as the "final translation." Is the goal a direct and literal translation even if it makes no sense or is grammatically incorrect?

December 29, 2012



A "none of the above" option would certainly be welcome. Often I find that the three sentences have factual or grammatical mistakes, so that I can't really endorse any of them as "good" translations.


Your suggested solution would encourage negative, unproductive criticism of others translations. The current system encourages productive criticism by allow one to pick the least worst option available and rewrite it entirely oneself by using the "Select and Edit" option. That's what I do, anyway.


I think that might happen as people who are confident of their own translation could discard anything that is not theirs. Discarding bad translations does not automatically lead to good translations as none of the answers might be satisfying to the corrector. I would still like to see some improvement to the current system as I've seen 100% mistranslations too.

I didn't use the "Select and Edit" feature more than once as it gave me the feeling that I was editing my own sentence, but perhaps I'm wrong. (ambiguous sentence) It would be nice if the editing feature would allow you to better select what parts of a sentence you like and what parts you don't. That might also make it less likely that people rewrite a sentence for its style as its parts might be well translated.

A separate discussion section with each sentence would be nice too. At the moment you only interact with three sentences and only one person gets a message if you suggest an edit, right? I would like some more interaction between the translators of an individual sentence as the translation process and discussions might help you to learn more about a language. At the moment you translate something and never hear anything from it unless your sentence is picked as the best or if someone has a specific suggestion for your sentence. That still doesn't say much as you remain unaware of the other sentences that people suggested.

I realise that this system might slow down the number of translations, but I think that the quality of the translations is just as important. Discussions may improve translations and users could at the same time improve their language skills. My suggestion for speeding up the process would be a timer for voting on suggested translations that starts running after a certain number of translators has joined the discussion. Maybe people could get some points for being second best too as a discussion could be split between opinions. The suggestion of a timer may not perfect as people could potentially join a discussion last minute to get some points for their sentence.

Sorry if my argument seems a bit contradicting. I tried to give an overview of the benefits and disadvantages of certain systems. I think that the separate discussion section could just be a field for people to place comments on a sentence (with a following option) while keeping the old system mostly in tact (I suggested that people could be able to say what parts of a sentence they like).


Excellent idea. Anyone familiar with psychometrics knows that adding 'none-of-the-above' options to a multiple choice test do a good job at discriminating the knowledge possessed by test takers.


That is what the " pick and edit " option is for . I agree that once it is " 100% " translated the " edit " option does not work .


Yes, please! A none-of-the-above possibility would be helpful.


On iphone app there is 3 marks: :( :| and :) for only one sentence - it's better solution


I think the scoring algorithm needs some tuning. In a recent document, I noticed that even though there was a person that had translated sentences before me, I was only confronted with the robot translations during scoring.


I suspect that may be because they're using us to test and rank different translation algorithms.


totally agree with you!


"None of the above" would, I think, help a lot.


I like the idea of adding "non of the above" It happens on a regular base that I am not satisfied by any of the translations so I refrain from commenting. Another strange thing is, that I sometimes get to grade my own translation. I am not comfortable with that, but do it and yes, in at least half the occasions I grade my own translation as the best. Another funny thing is, that Duobot is getting better at "his" job. It even happened, that I voted a duobot translation as the best (once).


I'm with siebolt. I recently chose my own translation on 3 occasions in a row, which I wasn't happy to do either, but I felt (as an English editor professionally) that I knew best. I would like someone else to make that decision. Some translations are so dire as to be beyond mending, but I have edited others' entries to improve them. The UK government won't allow "none of the above" in national and local elections because there is a huge demand for it!


Another possibility would be allowing you always to pick and edit your own translation as the best.


That would be alright for Anglosaxons with translating experience, otherwise it is an invitation to less conscientious people to play around. At least I would not be comfortable with having to judge my own translation every time. I think my translations are okay, but certainly not really good. On the other hand I think I can judge if somebody has done a good job so I keep grading.


I agree with siebolt. Even for non native speakers it is usually easy to spot poor translations but difficult to come up with really good ones.


That could work. But I suspect that every translator is going to be, to some extent, biased in favour of his or her own translations, so you would need some way to identify and account for that bias in the algorithms. Another idea (if you're not already doing it) would be to weigh votes for a translation on the basis of the average proportion of votes that a given translator's work recieves. Hence work from consistently good translators would be more likely to be selected than work by consistently bad translators.


Incorrect translations are not promoted, assuming duolingo has any sort of respectable algorithm.


I think the problem amenns is highlighting here is that users are sometimes de facto required to vote to select the best of three very bad - or, if you will, incorrect - translations and those votes are used to select the best translations. As a result, bad translation are being promoted. It's similar to the selection problem inherent in the US electoral system, whereby voters are de facto required to vote for either a Republican or a Democrat, even though they would often really prefer to vote for neither. She's pointing to a problem with the selection algorithm itself, and suggesting a solution.


First of all, you cannot know duolingo's algorithm without either collecting experimental data on the "best translation" result and rankings or obtaining access to duolingo's code base.

"As a result, bad translation are being promoted. It's similar to the selection problem inherent in the US electoral system, whereby voters are de facto required to vote for either a Republican or a Democrat, even though they would often really prefer to vote for neither."

It depends on the algorithm. If duolingo uses relative data to calculate the best translation, and what I mean by this is only improving a translation's rank if compared to higher-ranking translations, then ranking bad translations will not affect the final result. To put it in terms of your analogy, this would be like using nationalized IRV to select the best candidate, or translation in this case.


An alternative to "none of the above" would be a simple scoring system, say A, B or C where A meant the translation was good, B meant it needed some work and C meant it was a complete failure.

Also contributors who provided past translations which were high class should have their suggestions favourably weighted as they would be more likely to be correct.


People, you are supposed to pick which one is best even when there is the worst of the worst. Just do your best to pick which one is the closest one to be correct!


I completely agree, I've seen some pretty bad translations.


i agree... sometimes none of the translations are good and it is impossible to choose one and give it a "smiley face"... what about the sad face option?

Learn a language in just 5 minutes a day. For free.