Duolingo is the most popular way to learn languages in the world. Best of all, it's 100% free!

https://www.duolingo.com/JensDeriemaeker

Need help for master's thesis research (Lingot reward)

Hi everyone,

I'm a Belgian translation student and for my master's thesis I'm evaluating the translation quality of crowdsourced translations and comparing it to professional translations.

In order to do this, I've uploaded four short tourism texts to Duolingo, for translation from Spanish to English. Unfortunately, none of the texts were fully translated/revised :( This message is actually a cry for help: I'm looking for as many translators as possible. The more data I can gather, the more representative my research will be.

If you don't feel like translating all texts, please consider just translating at least one sentence, or only revising already submitted translations. This is extremely important to me, so I'd appreciate it a lot! Free lingots for everyone who helps and comments in this discussion ;-)

These are the texts:

4 years ago

55 Comments


https://www.duolingo.com/jkulifay

Duolingo has the theory that a group of people working collaboratively will enhance the translation. I think that's true. But in Duolingo, I've found many shortcomings to having accurate translations. These should not be underestimated.

  1. Many/most of the students are at a beginners level and not capable of giving a good translation.

  2. There is no requirement to finish a document.

  3. There is no requirement to check documents that others have translated.

  4. When Duolingo "thinks" that the document is sufficiently translated, based on the tier level of people working on it, the number of upvotes, and the similarity to duobot, then the document is highlighted in black. Once it is highlighted in black, translators assume that it is complete and do not work on it.

  5. The beginning of long documents are usually fairly well completed, but the end of long documents are rarely as thoroughly reviewed. You had a tough time getting sufficient translations for your short documents. My guess is that it would have been significantly harder to get long documents completed.

  6. Crowdsourcing on translations works better if an article is assigned to a team.

  7. Some of the people that work on the crowdsourced translations are capable of being professional translators, hence skew a comparison.

  8. Duolingo's reward system of receiving credit for a minor change in the previous person's post encourages revisions that are not meaningful.

  9. Crowdsourcing for translations is still "too young". When you review the number of people who are using Duolingo, how frequently Immersion is used, and the average level of the translators in Immersion, you will see that the level of "maturing" of the group of translators is still occurring. To get to a higher level tier, one must have their translations upvoted by others. Most people are at level 1, 2 or 3. As the people approach levels 5, 6, 7, the only way to advance to another level is to work in a small collective that checks each other's work. As time passes, you will have more people who are at higher levels and can upvote sentences. I think you will always have many more at the beginning levels, but as time goes by, you will raise more upper level tier translators.

  10. Duolingo does not have a mechanism by which you can see all the people who reviewed the article. When I upvote sentences in your translations, but do not edit them, my name is not listed in the translator section. There is no way for you to know that I've reviewed the sentences and found them satisfactory.

These were just a few ideas to keep in mind and items you may want to address in your paper. Good luck on your thesis.

4 years ago

https://www.duolingo.com/JensDeriemaeker

I am aware crowdsourcing for translations is still too young; part of my thesis will discuss its shortcomings. You make some very valid points, some of which I hadn't even considered yet, and I will surely adress these! Thank you so much!

4 years ago

https://www.duolingo.com/Markmcopc
Markmcopc
  • 25
  • 25
  • 25
  • 25
  • 25
  • 22
  • 5
  • 1924

In that case, Jens, may I please recommend the discussion The fault of learning? Quite a number of the issues raised by jkulifay are currently being debated there.

4 years ago

https://www.duolingo.com/JensDeriemaeker

Thanks again, Mark!

4 years ago

https://www.duolingo.com/Carolind
Carolind
  • 14
  • 12
  • 11
  • 10
  • 10
  • 5

Interesting appraisal

4 years ago

https://www.duolingo.com/Markmcopc
Markmcopc
  • 25
  • 25
  • 25
  • 25
  • 25
  • 22
  • 5
  • 1924

Thank you for inserting such a helpful summation of the state of affairs.

I agree with the implication that Duolingo is still in the early stages of working toward achieving the sophistication of other kinds of successful crowd-sourcing projects with similarly ambitious purposes.

4 years ago

https://www.duolingo.com/jkulifay

I can appreciate that a programme like Duolingo "can" achieve significant advancement in translations. Wikipedia transformed the world of encyclopedias. And in general the level of information entered is average or above average. In some cases, it is quite good.

4 years ago

https://www.duolingo.com/Luyematsu
Luyematsu
  • 25
  • 24
  • 14
  • 9
  • 9
  • 8
  • 55

I'm working on Costa Blanca and have left a note in the discussion tab. I'm not sure whether you, as the researcher, are allowed to provide context...who the intended audience is, whether each article stands on its own or is part of a larger work...which may affect how best to translate certain items.

4 years ago

https://www.duolingo.com/Carolind
Carolind
  • 14
  • 12
  • 11
  • 10
  • 10
  • 5

I think part of what you are saying here is the beauty of crowdsourcing. We fill in each other's gaps. The information you have about Malaga made for a better translation.

4 years ago

https://www.duolingo.com/Luyematsu
Luyematsu
  • 25
  • 24
  • 14
  • 9
  • 9
  • 8
  • 55

I think you meant to respond to lrtward, but it got attached to my comment instead?

4 years ago

https://www.duolingo.com/Carolind
Carolind
  • 14
  • 12
  • 11
  • 10
  • 10
  • 5

Sorry. You are right. Edit: I said you were right, but actually no. It was really meant for you. I just mixed up Malaga and Alicante. So I meant: the comment you made about Alicante made for a better translation. Does that make sense?

4 years ago

https://www.duolingo.com/Lrtward
Lrtward
Mod
  • 25
  • 25
  • 8
  • 8
  • 698

Carolind, you had some great input into the Costa del Sol article today. I almost wrote a note on your Activity Stream but got distracted. I think we stepped on each other's toes once or twice accidentally, when we were editing the same sentence at the same time, so I hope I didn't offend you with anything I did.

4 years ago

https://www.duolingo.com/Carolind
Carolind
  • 14
  • 12
  • 11
  • 10
  • 10
  • 5

Hey there! I am hardly ever offended in DL. I take this as collaborative work and learning. Even if in the end we don't get to an agreement on a specific point, it is still the discussion itself and how much a learn on the way that interest me. If you disagree with me, I'll never take it personally. I'll actually welcome the discussion! I hope you take it the same way. I always try to justify my choices too. Though I'm not always very clear in my explanations. If I did step on your toes, let me know when and where so I may try to make it up to you. Having said all that, I learned a lot from your translations too.

4 years ago

https://www.duolingo.com/Luyematsu
Luyematsu
  • 25
  • 24
  • 14
  • 9
  • 9
  • 8
  • 55

Thanks! That "provincial capital" seemed vague to me, so I had to figure out what was going on! I'm glad it was helpful.

4 years ago

https://www.duolingo.com/JensDeriemaeker

Yes, context may have an important impact on translation decisions. I did want to provide some context, but I wasn't sure how to do that here (I didn't want to provide the information in the document itself, because then that text would be included in the translation progress). Right now, I think it would be best to keep things this way, without any context.

4 years ago

https://www.duolingo.com/Sophieglot

I've translated and revised a little bit of them but it's probably not very good...

4 years ago

https://www.duolingo.com/JensDeriemaeker

That doesn't really matter, I appreciate the effort :) !

4 years ago

https://www.duolingo.com/Markmcopc
Markmcopc
  • 25
  • 25
  • 25
  • 25
  • 25
  • 22
  • 5
  • 1924

I am very interested in what comes of this project, and hope to see it succeed. I've asked some friends over. I hope they'll be interested too. Tot ziens, à plus tard, see you soon.

4 years ago

https://www.duolingo.com/JensDeriemaeker

Thank you very much for your cooperation! Great to see you're learning Dutch! I never had to learn it since it's my native language, but I do believe it's one of the hardest languages to learn, so best of luck!

4 years ago

https://www.duolingo.com/AureliaUK
AureliaUK
  • 25
  • 25
  • 25
  • 22
  • 14
  • 13
  • 12
  • 11
  • 8
  • 3
  • 2
  • 8

I've done a full review of Costa de la Luz with quite a few amendments and in-line comments. Hope that helps.

4 years ago

https://www.duolingo.com/JensDeriemaeker

Amazing work, I couldn't thank you enough :) !

4 years ago

https://www.duolingo.com/AureliaUK
AureliaUK
  • 25
  • 25
  • 25
  • 22
  • 14
  • 13
  • 12
  • 11
  • 8
  • 3
  • 2
  • 8

Interestingly this article seems to have attracted a troll - a level 1 translator who insists on reverting one sentence to someone else's incorrect translation even when the error is pointed out to him. This is of course exactly the problem that is likely to occur with crowdsourcing translations, and difficult to report as abuse as it's not clear that doing so would not end up penalising the person who originally made the error in good faith.

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

One needs to email abuse@duolingo.com directly in a case like this. The button to report abuse will, as you say, almost certainly report the person whose translation the troll selected.

4 years ago

https://www.duolingo.com/AureliaUK
AureliaUK
  • 25
  • 25
  • 25
  • 22
  • 14
  • 13
  • 12
  • 11
  • 8
  • 3
  • 2
  • 8

Yes, I have reported it. No idea if it will do any good.

4 years ago

https://www.duolingo.com/Carolind
Carolind
  • 14
  • 12
  • 11
  • 10
  • 10
  • 5

I have worked on Costa Blanca and I've been as thorough as I am with my favorite texts. I'll check out the other ones too. I am interested in the results of your research. Could you send me a link to an article when you write one? My Flemmish is still pretty basic, so if you have something in English (French, Spanish or Portuguese) that'd be perfect. (PS: Notice I said "Flemmish" and not Dutch. I learned the little I know in Belgium :-)

4 years ago

https://www.duolingo.com/JensDeriemaeker

Great! My thesis will be in English, so you needn't worry ;-) It is due on August 8th, and I'll only get feedback by the end of September, so I'll probably only be able to upload it by then. I did notice you said Flemmish, by the way ;-) Dutch and Flemmish are definitely not the same language :D

4 years ago

https://www.duolingo.com/wazzie
wazzie
  • 25
  • 11

If you were willing, I think it would be great if you could post (a link to) the article here. Most people are now 'following' this discussion, so we would get an email notification of the post (even if it isn't until September :))

4 years ago

https://www.duolingo.com/JensDeriemaeker

I'll be sure to provide you all with a link to my article when it's finished :) Thank you for your interest!

4 years ago

https://www.duolingo.com/Lrtward
Lrtward
Mod
  • 25
  • 25
  • 8
  • 8
  • 698

Costa del Sol is getting a pretty good vetting :)

I highly recommend that translators not only work on individual sentences, but we also click the middle button (review? check? proofread? I have my interface set to a language other than English at the moment) and read the entire article as a unit, and edit for flow, continuity, and consistency. This particular article has an abrupt and, to me, uncomfortable change from Malaga to Malaka and back, but the original does the same thing. I don't like it, but I didn't change it.

4 years ago

https://www.duolingo.com/JensDeriemaeker

Exactly, that is also what we are taught in school, to look at the text as a whole, not individual sentences :) However, most of Duolingo's users (or at least that's what I assume) aren't aware of this, they tend to translate seperate units. This is actually one of the shortcoming of crowdsourced translation which will be discussed in my thesis :)

4 years ago

https://www.duolingo.com/jkulifay

I think this is also a function of the level of Spanish fluency. When you first start, you translate word for word. Then you graduate to phrases by phrases. Next is more on the level of sentences. Lastly you mature to translating ideas.

This is one of the problems with crowdsourcing. First, an upper level translator may translate the ideas and meaning. Afterwards, a first level translator may come back and indicate that it didn't "say" that, and re-translate as word for word.

4 years ago

https://www.duolingo.com/Lrtward
Lrtward
Mod
  • 25
  • 25
  • 8
  • 8
  • 698

Agreed. Plus, with crowd sourcing you get people who may be interested in the culture, the slang, the ability to communicate... they are interested in learning the language, yes, but not necessarily interested in grammar, punctuation, and spelling. I mean, my Spanish is good but not great and I frequently see posts by native speakers that make me cringe due to misspellings, poor grammar, and lack of punctuation and capitalization. So I really doubt they'd be a first rate translator.

And even the folks like myself, who tend to be pretty anal about grammar, spelling, punctuation, and related mechanics of a language... well, we don't always make good writers/translators. We might not catch double entendres, attitudes, or the color of an article - that ineffable quality that makes a piece of writing a tapestry rather than a polaroid. And even if we DO notice that type of thing in the secondary language, we may not be capable of bringing it across effectively into our native language.

That said, most of the crowd sourcing I've seen come out of Duolingo is almost as good as what I saw on a Korean tourism website (in English) recently :) I think really good translation requires a unique set of skills, talents, and experience.

That said, put me down as another person who would love to see your paper when you're done.

4 years ago

https://www.duolingo.com/ViejitaKatalina

Yes, I'm a "Level 1" translator because I'm relatively new to Duolingo, but I've studied Spanish literature in Spanish at the university level. But I'm rusty -- and I have no delusions of being an amazing translator. So, I appreciate the checks and balances of crowdsourcing. At the same time, I have noticed that my translations sometimes have been changed into something more literal.

I'll take a shot at your pieces.

4 years ago

https://www.duolingo.com/mjstallworth

I've added what little I could. Hope it helps!

4 years ago

https://www.duolingo.com/JensDeriemaeker

Much obliged!

4 years ago

https://www.duolingo.com/mjstallworth

Thanks for the lingot! I didn't do much. I hope you get the answers you need.

4 years ago

https://www.duolingo.com/Lavinae
Lavinae
Mod
  • 25
  • 25
  • 20
  • 13
  • 12
  • 12
  • 11
  • 11
  • 9
  • 8
  • 7
  • 7
  • 7
  • 3
  • 2

Since they're short texts, I'll go over them sporadically today (and tomorrow) :)

Succes met je scriptie! Ik zit in hetzelfde schuitje... ;)

4 years ago

https://www.duolingo.com/JensDeriemaeker

You are very much appreciated ;-) En ook heel veel succes met je scriptie!

4 years ago

https://www.duolingo.com/Aliciaftw

This sounds extremely interesting. Good luck on your thesis. =) When is all this due by? I'll take a look tonight.

4 years ago

https://www.duolingo.com/JensDeriemaeker

It's due on August 8th, so no need to rush :) Right now, I'm still working on the theoretical framework, but I'd like to start analysing the results of this little experiment in two weeks or so.

4 years ago

https://www.duolingo.com/Spaceowl42

I help mark all the unchecked sentences! :-D Happy learning! - SpaceOwl42

4 years ago

https://www.duolingo.com/JensDeriemaeker

Thanks!

4 years ago

https://www.duolingo.com/Spaceowl42

Your welcome! (¡Da nada! ;-D)

4 years ago

https://www.duolingo.com/hudsodm

I translated Costa Blanca from scratch and reverted my translation to the previous crowd-sourced version. It was a much freer translation than I would normally do. If I have time to look at other texts, would you prefer that I proceed in the same way or just proof and edit the existing translations?

--Don

4 years ago

https://www.duolingo.com/JensDeriemaeker

Thanks a bunch! You can proceed in the same way :) Right now, three of the four texts have been fully translated and revised; only Costa Blanca still needs some proofreading.

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

Okay, check it now.

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

I think they're all done now, do you agree?

4 years ago

https://www.duolingo.com/JensDeriemaeker

There is only one more sentence in Costa Blanca that needs revision :)

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

From the perspective of your thesis, you probably want to know that Mar Plateado "adopted" your documents.

https://www.duolingo.com/comment/3013480

You might be interested in our process, since it suggests a way to make crowdsourced translations more effective.

4 years ago

https://www.duolingo.com/AureliaUK
AureliaUK
  • 25
  • 25
  • 25
  • 22
  • 14
  • 13
  • 12
  • 11
  • 8
  • 3
  • 2
  • 8

I wonder if it would be more instructive for MP to adopt only two of the articles, as it might be possible to see a real difference between the output of random individuals -v- the output of a team of peers.

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

Actually, the fact that he posted a personal plea for help has, to some degree, distorted his results already. His documents already got a great deal more attention than the average submissions to Immersion. Accordingly, I would say that what's he's getting is an estimate of the best that crowd-sourced translation can be. That being the case, I think we ought to weigh in.

4 years ago

https://www.duolingo.com/Markmcopc
Markmcopc
  • 25
  • 25
  • 25
  • 25
  • 25
  • 22
  • 5
  • 1924

I think that the sample of the range of results that arise from driverless mode is large enough to make a meaningful comparison. Sometimes better,sometimes worse. MP is predictable, not necessarily the best every time.

And it is the best model for developing the trusting relationships that make possible improvement over time. I would love them not to kick me out!

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

You definitely put your finger on the key point; to be effective, a team of translators needs to trust each other. I can tell Aurelia "I reverted your edit for reason X" and at the worst we'll discuss it a bit. Strangers sometimes take that well, but (as we saw even in these four documents) sometimes they just blow up.

What's this about someone kicking you out of something?

4 years ago

https://www.duolingo.com/Luyematsu
Luyematsu
  • 25
  • 24
  • 14
  • 9
  • 9
  • 8
  • 55

Probably too late for that. Greg seems to be up to full speed again! :)

4 years ago

https://www.duolingo.com/GregHullender
GregHullender
  • 25
  • 25
  • 22
  • 13
  • 10

Oops . . .

4 years ago