1. Forum
  2. >
  3. Topic: Duolingo
  4. >
  5. Course Contributor AMA!

https://www.duolingo.com/profile/Trofaste

Course Contributor AMA!

Good morning/afternoon/day/evening or night language enthusiasts!

This AMA is now closed. Thanks to everyone who participated! The post will now be locked to avoid the frustration of people ending up in this thread and never getting their question answered.

From 20:00 (8pm) UTC on 8 September to 20:00 (8pm) UTC on 9 September *, I (Trofaste) will be here to answer all of your questions about course contributing, as a course contributor and, of course, as a regular Duolingo user as well!

You're free to ask anything you like! However, I reserve the right to not answer personal or confrontational questions, or questions that make me uncomfortable. Furthermore, questions regarding the release of courses into or from beta, the addition of new languages to the Incubator, the addition of new features, or the expansion of existing features to more languages might receive a standard answer: my knowledge regarding these topics is limited, and in the rare event I know something you don't already, I wouldn't be able to tell you or you'd already have heard it. This doesn't mean that asking these questions is not okay: these questions are as legitimate as any other. What it does mean is that my answers might not be very insightful and that these questions will likely receive a similar if not standard reply.

Also, I can't reveal every detail of the inner workings of Duolingo and the Incubator. I can only share screenshots that have been approved by Duolingo staff and only give details that staff have cleared me to give. You can ask, but be aware that I may not be able to give you as full an answer as you might like

I'll still be very grateful to receive all of your questions, but there are some questions which I'll be able to answer better than others. Please bear with me. :)

And lastly I would like to thank all of you for your appreciation for us and for the constant help we receive through your reports.

So with those side notes, ask me anything!

Trofaste

*Questions can be asked ahead of the AMA. However, they will not be answered before it starts.

PS: I am planning to sleep while this post is up (after all, it will be open 24 hours!). Don't worry if you don't get an answer to your question immediately, I will get to it!

September 7, 2019

186 Comments
This discussion is locked.


https://www.duolingo.com/profile/Jan139433

Do the contributors come up with a theme or organizing approach to a language? Or is that done by staff? For example, Latin has some running jokes about drunk, angry parrots that appear in many lessons. Who decides to do that? Is there a "writer's room" where people brainstorm ideas and themes?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

Contributors do that. Sometimes it's coordinated by the team, sometimes it's user inspired (possibly without the user's knowledge), sometimes it just kind of happens. But it makes the course creation much more fun for us too! It can be a rather mundane task sometimes, but those things spice it up a little and keep it interesting. It's usually just done by the team but occasionally discussions do happen... But I might have to kill you all if I revealed the things we talk about behind the scenes. ;-)

September 8, 2019

https://www.duolingo.com/profile/duolingouser621

What do the reports we send look like inside the Incubator? How are they organized and what do you actually do with them?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

LICA98 is right that we're not usually allowed to share screenshots, but I have special permission to share some staff-approved ones. :)

User-suggested translations

This is a set of "My answer should be accepted" reports, or as we know them, user-suggested translations. They are a few of the reports (only a small set, but representative of the rest) from the reverse exercise (translation from the source language to the target language) on a sentence in the English for French speakers course. The French sentence being translated is "Ce sont des femmes." and the English translation is "They are women." The number in the circle is the number of times that answer has been reported (since the last time someone deleted it). Then there's the text of the report, and the buttons for handling it are off to the right, not visible in the screenshot. For each report, we can reject it, which deletes it from the list, or we can accept it, which sends an email (but it has to be manually added to the list of accepted answers). Once in a while someone accidentally clicks the accept buttton on a very wrong report. It sends the email, but don't worry, it's not actually been added!

freewrite reports

This is a set of "freewrite reports" for the same sentence. These are all shown in a group beneath the user-suggested translations (the other standard report types are shown in a group below the freewrites). These are currently only available on Android. We can only read them, and then delete them when we've got what we can out of them, there's no way for us to respond to them. Like the user-suggested translations, most of them are pretty useless, this screenshot is representative. The first is apparently a complaint about the speech recognition, which we can't do anything about. There's a question, which we can't answer, a declaration that "there is a problem" which is pretty useless without more information, ditto for "the sentence is not correct". "They are woman" is probably intended to tell us that that's what the translation should be, which is wrong and should have been reported with "my answer should be accepted" even if it were right. The rest are just clutter which wastes our time, as you can probably tell even if you don't speak French.

September 8, 2019

https://www.duolingo.com/profile/jairapetyan

Wow, that's an eye-opener. I will never lobby for freewrites again. Of all the suggested translations, there is actually one in the top list that is acceptable in colloquial American English. But in general I feel sorry for you that you have to go through such amounts of mistaken "correct" sentences. Luckily I have an eye for detail and it's really rare I would make a typo.

September 9, 2019

https://www.duolingo.com/profile/Larisa_L

The free-write reports are 99.9999% useless and even if there is anything useful, then it definitely ends up in the "suggested translations section". So there is no real need to read those free-write reports. I've had only a handful of cases where a message from a user actually helped me to figure out what was wrong with the sentence. But then, there are plenty of times when you get offensive and very offensive messages, so better not to read them most of the time.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I usually give them a quick read through (sometimes there can be clues that we need to teach something better in the next tree version, for example), and then delete them all. Because yes, they're most often useless.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

@Mereade It's been suggested before, but personally I don't want users to be able to see all the translations. We choose the best translations and leave the others only as accepted for a reason. Often there's an answer we have to accept because it's technically correct, but we don't want to encourage it (when I add those I even click the reject button on any reports because I don't want to send an email about it). Also I'm not eager to face all the criticism there would be from people who don't like the format we write things in, or think we shouldn't accept this or that or the other who wouldn't have a clue otherwise. And frankly, I doubt it would get much use of any value to us except from users who already know not to report unless their answer doesn't have a typo or anything.

September 9, 2019

https://www.duolingo.com/profile/Mereade

This is really a very good piece of information, thanks!

I wonder: perhaps the quality of "my answer should be accepted" reports could be higher, if we got to see the alternatives too? When I answer, I could be shown not just one alternative (or zero, depending on whether I've hit the main answer), but the list of alternatives? Perhaps that would help people notice that their alternative is there, just without the typo, etc? I think it would also have a big educational value.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I'm actually not sure which one you're referring to... "They are womans"? There are also a few that are fine sentences, just not as translations for that particular sentence ("They are the women", "Are you women?" "There are women").

September 9, 2019

https://www.duolingo.com/profile/kaet
  • 159

Thanks for this! What are the percentages in the first picture?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I knew someone would ask that. :-) That's a new feature, it's how likely the system thinks it is that the report is accurate. It learns from what actions we take on the reports, and since it's new, it has a lot of learning to do, so it's not a great deal of use yet.

September 9, 2019

https://www.duolingo.com/profile/hanspersson

I see I'm not the only one to mistakenly type in the wrong language (in my case usually German when doing EN->FR, since I'm doing DE->FR in parallel) but I don't think I've ever taken it as far as reporting it without realizing that I'm thinking in the wrong language...

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes, that happens quite a lot. I've even seen reports from people who know they wrote in the wrong language and want it accepted anyway (this actually happens a lot in the Czech<->English courses, where people want us to accept Slovak). It's also not uncommon for one word to get switched by mistake, and even I have been guilty of sending reports like that and realising after the fact that something was another language... After reading my sentence carefully too, even checking my gender agreements!

September 9, 2019

https://www.duolingo.com/profile/duolingouser621

How difficult is it to keep courses consistent, especially when there are a lot of contributors?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

It's hard to quantify difficulty, but it can be pretty tricky! And usuallt the more contributors there are the harder it is. It's also harder when there's a reverse course, because sentences and translations are in a shared database and editing a sentence in one will change it in the other as well. This sometimes results in accidental edit wars... The best way to maintain consistency is to make sure that conventions are established and documented, and discussed in a place where all the contributors are active. Having reverse courses have the same contributors helps too, as does keeping teams smaller. That comes with fewer people helping with the work, so either work not being done as fast or the contribs working more, so you have to balance it. A small team of highly active prople has generally proven to be quite effective, though.

September 8, 2019

https://www.duolingo.com/profile/jairapetyan

That's a pity that reverse courses can't have a database of their own. I can understand using the original course as a framework, but then separating them. I'm sure there are lots of considerations when trying to find the right balance between colloquial and literal translations, multiplied if you have to do it two ways!

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes, it's a tricky balance to find. And there are always users unhappy with whatever we decide.

September 9, 2019

https://www.duolingo.com/profile/Larisa_L

Yes, it is so tricky, that in our Eng-Rus courses we even aim to not reuse sentences. If I see that the sentences is being used in the Russian course (not always obvious), then I delete it in the English course and replace with some other sentence.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes, not reusing sentences helps, but that can be hard to avoid, especially early in the tree. Sometimes you're just stuck with it.

September 9, 2019

https://www.duolingo.com/profile/hanspersson

Oh? I thought reverse courses were totally separate from their counterparts since they are pretty different, but apparently not.

September 9, 2019

https://www.duolingo.com/profile/jrikhal

Hi hanspersson,

the list of skills, of lexemes and of sentences in the tree are independent.
But if a sentence happens to be in both then they share a same unique list of accepted translations.

And, hints are also shared.

September 9, 2019

https://www.duolingo.com/profile/hanspersson

OK, that makes sense.

September 9, 2019

https://www.duolingo.com/profile/MopasMipy
  1. What's contributing, exactly? Do I have to provide 30 translated sentences a day or what?

  2. Can you "contribute" stories? (Even one set?)

  3. Do contributors need to translate certain words/sentences? (like how "bread", "apple" is pretty much in most of the languages I tried out so far.)

  4. If there are no language courses from a certain language, can one apply to contribute to making a course from that certain language?

  5. Can the first course to a new language not be English? (if there are more contributors for that, obviously.)

Thanks for taking the time to read my questions, no need to answer them all ;v; GL! :D

September 7, 2019

https://www.duolingo.com/profile/Trofaste
  1. Contributing is creating the course content. That means planning the course, adding the skills, adding the words to the skills, writing the sentences, adding the translations and hints to those sentences, checking the audio (or recording it, depending on the language), adding image exercises, writing tips and notes, etc. Then after the course is released it means responding to reports, so primarily adding missing translations and removing wrong ones that snuck in, and writing new sentences. Then after the cohrse has matured some, creating a new tree, which means doing the first part again while continuing to.do the second part on the live tree.

  2. No, the contributors have no connection to stories.

  3. Courses generally more or less follow the CEFR guidelines, so while we aren't given any specific requirements, there are a lot of words that will be common to many courses. The same words are used early in multiple courses because they're often simple and useful for illustrating grammatical points without overwhelming learners. Sentences have to use only the words and grammar already taught, so if the early words are similar the early sentences will be similar too.

  4. You can certainly apply, if you're fluent in both languages and able to commit significant time to the course.

  5. There's no reason it can't be. It would depend on what course would have the most potential benefit, the difference in cost to Duolingo, and the other factors Duo considers when choosing what courses to add.

September 8, 2019

https://www.duolingo.com/profile/jairapetyan

What costs does Duolingo have to shoulder, given that the linguistic part is provided by volunteers?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Server load on the (already appallingly slow) Incubator, and costs of supporting the team, especially technically (some languages in particular require a fair amount of technical support, like Arabic or Japanese) would be the main ones.

September 9, 2019

https://www.duolingo.com/profile/MopasMipy

Thanks so much for taking the time to answer! I might try contributing soon, maybe in the holidays or so, we'll see ;)

(Another Q: Can I contribute anonymously?)

GL! :D

September 9, 2019

https://www.duolingo.com/profile/Trofaste

No, if you're a contributor you'll be listed on the course's status page in the Incubator for all to see. But nobody has to know your real name, if you don't want them to.

September 9, 2019

https://www.duolingo.com/profile/MopasMipy

Thanks for letting me know. If that's so, I'd probably contribute from another account. GL! :D

September 9, 2019

https://www.duolingo.com/profile/Finlay745011

catalan is not in english, so you can have the first course for a new language not in english

September 8, 2019

https://www.duolingo.com/profile/MopasMipy

I meant that the first course from a language would be not English (ex. if there were no courses from Spanish, Catalan from Spanish would be first & not English from Spanish) but you're right.

GL! :D

September 8, 2019

https://www.duolingo.com/profile/J-apprends

Can you do anything about bad audio? What about the dictionaries? And can you add any sentence you want to courses?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

Can you do anything about bad audio?

That depends on the course. If it's a course with recorded audio, yes, it can be re-recorded. If it's a course with TTS, though, all that can be done is disabling the "type what you hear" exercises.

What about the dictionaries?

No, contributors don't have control over those.

And can you add any sentence you want to courses?

Yes, with limits. Anything that wouldn't be allowed on the forum is not allowed in the courses. That's not usually an issue though, as most contributors have common sense! The main limitation on the sentences we write is the words in the course. Sentences can only use words and grammar taught in the course. And we have to write a minimum of three sentences for each word where we introduce it, which can only use words and grammar taught in that lesson or earlier. That can make it very difficult at times. If anyone wonders why the first couple skills are all rather similar and rather boring, it's because we can't do much else! Imagine you can choose eight English words. You have to be able to write at least 3 sentences for each of them, using only those eight words, and you need it to be simple for someone completely new to the language and be a good building block for later. It's not easy, is it? And 8 words is generous, 6 or at most 7 is better.

September 8, 2019

https://www.duolingo.com/profile/Songve

Two questions.

  1. What are the prerequisites for a course contributor?

  2. What are the expectations of a person accepted as a course contributor, such as minimum hours expected to work on the language and respond to Sentence Discussions?

September 7, 2019

https://www.duolingo.com/profile/Trofaste
  1. Contributors should be fluent in both languages (as the application page says, you don't have to be a native but you should speak at the level of one). You should also understand the grammar of both languages (being a native speaker is not enough, unfortunately many a native writes very badly in their own language). You don't need to have a degree, but it would certainly be a plus! Skill with technology would also be a plus, the Incubator is a tricky tool to learn and use.
  2. First, answering questions in SDs isn't actually part of ghe contributors' tasks. The forums are for the users, by the users, and it's as much your job as ours to answer the questions you're able to. When we answer questions, we're doing so as normal users, not in our capacity of contributor or moderator. :) Now to actually answer your question, there's not a set minimum, and needs will vary from course to course, and depend on your role. For example, someone who joins to write Tips & Notes won't need to commit as much time. But 3 or 4 hours a week isn't going to cut it. Applicants should be expecting to spend more like 10 hours a week, at least.

You might also be interested in a comment I wrote here describing much of what I'm looking for in an application.

September 8, 2019

https://www.duolingo.com/profile/nueby

First, answering questions in SDs isn't actually part of ghe contributors' tasks. The forums are for the users, by the users, and it's as much your job as ours to answer the questions you're able to.

Perhaps this should be posted somewhere more prominent to help users with imperious tendencies avoid going down the wrong path.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

One of these days I may write a post about what the contributors' and moderators' roles are not...

September 9, 2019

https://www.duolingo.com/profile/KaptianKaos8

Is B2 in a language not enough to contribute then?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Typically, no. In some circumstances a team may look for someone who may not be at as high a level in one of the languages, but unless they make a post detailing what they're looking for or headhunt you, assume you need to be fluent.

September 9, 2019

https://www.duolingo.com/profile/Algarve11

Sorry for busting in, but... I am a Contributor for the English-Russian course even thought I have only B2 in English. I would say that it depends on how much and what sort of work needs to be done in the course in which you want to Contribute. As a rule, it's better to have C2, but there are exceptions.

September 9, 2019

https://www.duolingo.com/profile/Larisa_L

Expanding on Algrave's comment. A lot of work in the course is done in 1 language, as we need to add thousands and thousands of alternative translations. For instnce, in our Eng-Ru course we are translating a lot of simple English sentences into Russian, and need thousands of Russian translations which would take into account all possible ways to say the same thing in Russian. You need to really know Russian well to do that, not English. Then there comes the reverse translation, from Russian into English. Now one needs to know English well. If one knows both languages well enough they can do both parts. Or the team lead can find 2 people to do each part separately. We did that a lot on our new tree, I found some English language natives to write English translations and Russian native speakers to write the Russian translations.

September 9, 2019

https://www.duolingo.com/profile/endless_sleeper

Which one is your (most) favourite Duo sentence?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

You would ask that question. :p It's really hard to choose... But if I had to choose just one, the one someone mentioned yesterday would have to be way up there, for its brutal honesty.

"Potřebujeme ještě dvě věty o Matějovi a tři o Žofii."
Translation: We need two more sentences about Matěj and three about Žofie.

Struggles of a course contributor... For the uninitiated, writing sentences can be a hard task. You have to find ways to use certain words in certain grammatical constructions without using words or grammar you haven't taught yet. It can be really difficult.

September 8, 2019

https://www.duolingo.com/profile/nueby

No fair. The someone was the author of that sentence and he mentioned your past comments about that sentence as evidence of your questionable taste in sentences. That should disqualify that sentence, and you owe us another!

September 8, 2019

https://www.duolingo.com/profile/Trofaste

Perfectly fair. Your mention of my past comments about it are evidence of its claim to the position.

Besides, it would be too hard to choose another... But "The girls are eating butter." might be an option, due to the genuine outrage and hysterical freewrite reports it generates! I've never been quite sure why people assume from that sentence that it means they're eating plain butter with nothing else, but they clearly do...

September 9, 2019

https://www.duolingo.com/profile/Larisa_L

Yes, we have the same with:

I eat salt. (Everyone eats salt, but people imagine someone actually eating it with a spoon or something).

Then:
Whose horse eats meat? (there are myths about meat-eating horses, but in any cases, the question doesn't imply that there needs to be any horse like that).

Not related to food and most surprising for me is:
He and my father are brothers. A pretty simple statement about your family, but so many people just cannot get the meaning of it. I cannot figure why.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I am strangely unsurprised that nobody understands the last one, though equally puzzled as to why...

September 9, 2019

https://www.duolingo.com/profile/Zia177448

That one is hilarious! XD

September 8, 2019

https://www.duolingo.com/profile/jaja889583

In the portuguese course I have had sentences like "They throw a cat to her", or: "Its nose touches my shoe", or: "the monkey drives the car" and such weird things. It is funny, eventhough sometimes hard to understand what is happening there.. I wonder if other languages have those strange sentences too?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Every course has its share of odd sentences. :)

September 9, 2019

https://www.duolingo.com/profile/Nanase4367

Hello Trofaste,

The Chinese for Japanese speakers course was added into the incubator at the end of July, but many weeks have passed and as of today the course has yet to have its first contributor and no one has stepped up to apply to contribute. It has been neglected for too long, and I am concerned the course could get removed from the incubator because of no contributors or activity. Is Duolingo currently working on finding contributors to work on the course, whether it's volunteers or Duolingo staff? Action is desperately needed and the course shouldn't continue to be left neglected and untouched for a long period of time.

Thanks.

September 7, 2019

https://www.duolingo.com/profile/Trofaste

I don't know anything about the Chinese for Japanese course, I'm afraid. Staff is likely looking for contributors, but I can't say for certain. I'm sure there have been applications, but whether they're any good or not is another matter... The majority of applications are sadly not up to scratch.

I don't think the course being removed is a danger, though. No course, however long inactive, has ever been removed, apart from April Fool's jokes a few years ago and one that was only added by accident.

September 8, 2019

https://www.duolingo.com/profile/amyleebell

Tagalog was in the Incubator a while back. I really hope that one gets made eventually, but apparently there are too many variations when translating from English to Tagalog. It was probably just too overwhelming and tedious.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Tagalog for English speakers has never been in the Incubator as far as I am aware. English for Tagalog speakers has, and it's still there.

September 9, 2019

https://www.duolingo.com/profile/SmashCookie

What's the meaning of life for a course contributor?

How's the daily live in the tribe of course contributors?

Is Duolingoism the official religion of course contributors?

What's the official dance of the course contributors?

September 7, 2019

https://www.duolingo.com/profile/Trofaste
  1. 42, of course!
  2. Coffee, clear reports, coffee, work/study/other real life, clear reports and eat and coffee, work/study/other real life, coffee, eat, clear reports, coffee, clear reports, clear reports, sleep for a couple hours (dreaming of clearing reports), repeat.
  3. For the Czech team it's Eclipsism...
  4. This.

https://giphy.com/gifs/dance-spooky-boi-bz7Lkscqm6mrK?utm_source=media-link&utm_medium=landing&utm_campaign=Media%20Links&utm_term=https://giphy.com/gifs/dance-spooky-boi-bz7Lkscqm6mrK

September 8, 2019

https://www.duolingo.com/profile/ehartz

No lies here. I've literally had frequent dreams (nightmares?) of adding alternate translations/clearing reports during phases when I've done a lot of work in the incubator...

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Hi eharts, hope it's ok to ask you a question here as you're a contributor? Which courses have you contributed to and which are you currently contributing to? Just wondering which courses it is that you've been having to add translations and clear out reports from? :-)

September 9, 2019

https://www.duolingo.com/profile/ehartz

My main course is Japanese for English speakers. I also hopped into the Latin for English speakers course recently just to help with adding missing translations for the launch while some of the real Latin speakers were busy, but I'm not actually all that proficient in Latin, so I don't consider myself a real contributor of that one. ;P

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Thanks! :-)

September 9, 2019

https://www.duolingo.com/profile/BoneheadBass

What you said plus...

I spend a lot more time in the SDs than in the Incubator. But I do a fair amount of putzing around in there when something that comes up in a forum requires it -- or when I need a break from user questions, and clearing reports seems like a better option. Then there are the steepness nights when I'm turning something over in my head for what seems like hours.

Finally, on a marginally related note, I love being part of the Czech team!

September 9, 2019

https://www.duolingo.com/profile/endless_sleeper

And we love having you (and TRB) around :-)

September 9, 2019

https://www.duolingo.com/profile/ISpeakAlien

Is Duolingoism the official religion of course contributors?

(Joke)

No, just the staff. They want it to be the official religion of the users.

September 8, 2019

https://www.duolingo.com/profile/J-apprends

Do you usually get results quickly when you ask staff to improve your tools? Compared, say, to how long it takes for learners to get changes we ask for.

September 7, 2019

https://www.duolingo.com/profile/Trofaste

It depends. If something breaks and we need it fixed urgently (meaning it doesn't work at all, if it partially works that doesn't usually count), staff is pretty on top of that. Anything else, though, is usually pretty slow. Just like for the rest of Duo, dev time is limited, and the Incubator is also running on old code. Many features we've asked for have taken years to get, or haven't happened yet, or have only partially happened. But we do get improvement now, even if it's very slow.

September 8, 2019

https://www.duolingo.com/profile/LingualKnuckles

Are you planning on adding more indigenous or endangered languages in the incubator soon? Having a Pipil (Nahuat El Salvador) or at least a Classical Nahuatl course is my dream.

September 7, 2019

https://www.duolingo.com/profile/Trofaste

I can't say what the plans are for new courses. Staff decides that, and we don't find out till they do (and if we did find out before you, we couldn't tell you anyway). It wouldn't surprise me to see more, though.

Edit: Just to make sure it's clear, I'm not a staff member. Course contributors are unpaid volunteers. :)

September 8, 2019

https://www.duolingo.com/profile/logane616

Hello, Trofaste. First of all, props to you for all your hard work. We appreciate it.

My question is this: how do you deal with the reports that come in? If, for instance, the report is about bad audio, do you have to record a new bite? If it is about the answer being obscure/incorrect, how do you fix it?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

My answer to this question might partially answer that. :) With user-suggested translations, we check if it's correct. If it is, we manually add it to the list of accepted translations (along with any others we see are missing). Then we click a button to get rid of the report, either accepting, which sends an email, or rejecting, which does not. For freewrites, we read them, if there's any information of value we act on it, then we delete. For audio reports, we listen to the audio. Most of the time there's actually no major problem and it's just that the learners ears aren't used to the language. When there is, if it's a recorded language, it can be re-recorded. For TTS languages, all we can do is disable the type what you hear exercises. For hints reports, we check the hints. Most often there's either a hint missing for the sentence, the automatic ordering has put the wrong one on top, or the user tried to ise a hint needed for another sentence on a sentence where it doesn't apply. The reports about the sentence being unnatural or having an error are too vague to be much use, but we'll check the sentence and best translation to make sure there aren't really any errors or horribly unnatural language before deleting. If there are, it's usually already been covered by the user-suggested translations and freewrites.

September 8, 2019

https://www.duolingo.com/profile/kaet
  • 159

How fluent do contributors have to be in the two languages? How much does this vary by language? I presume this will be different for Latin than for a fully living language, and might also depend on the number of volunteer contributors. How are volunteers tested for dual fluency?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

As the application page in the Incubator says, you don't have to be a native speaker, but you should speak at the level of one. This especially applies for new courses. Once there's an established team, they may look for someone who's not as good in one of the languages but exceptional in another to fill their holes in ability (eg. if you have two contributors, both fluent in one and native in the other, it can be valuable to have a native speaker of the one even if they're not fully fluent in the other) or to perform a specific task (eg. writing Tips and Notes). But unless the team posts a request for applications stating what they're looking for or headhunts you, assume you need native level in both languages.

How are volunteers tested for dual fluency?

I don't know how staff does it for the first contributors to a course, I've never been involved in that process either as helping choose the contributors or as the one chosen, I've only joined existing courses. For existing courses, the contributors have the application to look at (you have to write a text in one language and then translate it to the other), but the most helpful information comes from their forum activity. That gives much better opportunity to assess their ability, and also lets us look at things like "will they stick to it". This is one reason why we generally choose people who are active in the sentence discussions.

September 9, 2019

https://www.duolingo.com/profile/kaet
  • 159

Thanks!

September 9, 2019

https://www.duolingo.com/profile/BoneheadBass

Regarding "bad actors"...

  1. How do contributors/moderators identify, er, "troublesome" users?
  2. Are there clear guidelines for dealing with them?
  3. If there are, can you give some examples?
  4. If you reveal any of this to us, will you have to kill us?
  5. Thank you for all you do, and for creating this post!
  6. PS - TRB sends his warm and fuzzy regards.
September 8, 2019

https://www.duolingo.com/profile/Trofaste
  1. Usually we know them when we see them, but we can't do much about them until they break the guidelines.
  2. There are.
  3. You can see the Moderator Guidelines here.
  4. Yes, definitely.
  5. You're welcome, and thank you too!
  6. Please forward my warm regards to TRB!
September 9, 2019

https://www.duolingo.com/profile/endless_sleeper

How many courses do you contribute to? I've lost count.

September 8, 2019

https://www.duolingo.com/profile/Trofaste

So have I. But of those I'm shown as a contributor in the Incubator it's hard to say which ones I count as actually contributing to...

For those who don't know, I'm on a lot of teams in a variety of roles. I don't actually speak all those languages (sadly)! It has happened that I've joined a team only to try to figure out a bug, or to teach that team's contributors how to use the Incubator.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

This AMA is now closed. Thanks to everyone who participated! The post will now be locked to avoid the frustration of people ending up in this thread and never getting their question answered.

September 9, 2019

https://www.duolingo.com/profile/dreamloomer

What is Duolingo planning to do with the courses that have been deserted by contributors? Danish course seems completely forsaken. Some time ago a contributor asked the users for their ideas, but then (s)he dissapeared completely. There are some mistakes in translations, a lot of good suggestions haven't been approved for years, but no one is taking care of it all. Is there any future for this course?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

If a team goes completely inactive for a long time staff will likely try to revive it eventually. But do bear in mind that a lot of what we do isn't visible to users, so a team may look inactive when they're actually working hard. There are definitely inactive teams, but don't be quick to assume that any particular team is one of them. :)

September 9, 2019

https://www.duolingo.com/profile/E.T.Gregor

What exactly does "revive" mean? Contact the contributors to ask them to continue or will DL eventually just pick new contributors? Also, how long is "eventually"? Some courses seem to have been inactive for years without DL staff acting. Can I assume that the contributors aren't as inactive as it seems to users or is DL just extremely patient?

Edit: For example, can you tell us what happened with the English for Telugu course? You're a contributor now, but if I remember correctly it had no contributors for quite a while. That probably counts as inactive ;)

September 9, 2019

https://www.duolingo.com/profile/Trofaste

That would mean removing the old contributors if they're unresponsive and finding new ones. I can't say how long staff would leave a course completely inactive before doing something, and it might depend on the circumstances.

I can't say anything about specific courses, but I can say there are definitely several courses that the teams seem inactive but aren't really.

September 9, 2019

https://www.duolingo.com/profile/E.T.Gregor

Thanks for the answer.

September 9, 2019

https://www.duolingo.com/profile/garpike

Who is the parrot-obsessive on the Latin team?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I don't know, but I suspect Nancy227353...

September 9, 2019

https://www.duolingo.com/profile/garpike

Please thank her on my behalf; the sentences are a lot of fun! Having running jokes is a marvellous way to make a language course more engaging than it otherwise would be.

I know Latin has only just been released, but may I ask if there are already any plans to expand it?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes. The usual pattern now is to start small, then iterate, making adjustments as we learn what can be improved from the learners and adding new content. The Latin course is no exception.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

@garpike All I did on the Norwegian tree was alpha test part of it and put up with Deliciae moving things around on me. And possibly inspire a few sentences, but I admit nothing. ;-)

September 9, 2019

https://www.duolingo.com/profile/hanspersson

The contributors to the Latin course are incredibly fast and effective. I assume they don't have time to work on extending the course right now given the time they apparently put into handling user reports, but I hope they'll get there in a while.

September 9, 2019

https://www.duolingo.com/profile/garpike

I know; I've had scores of reports back. I was only asking, not complaining. I think the Latin course has been put together really well; there just needs to be an awful lot more of it. I was not being critical. The contributors have done an excellent job, so far.

However, Trofaste says 'yes', and Trofaste is, at least partially, responsible for the enormous Norwegian tree, so I have high hopes for this course.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes, it will have to wait until the course is more stable. Usually new trees aren't started until a course gets out of beta, and not always right away after that.

September 9, 2019

https://www.duolingo.com/profile/kaet
  • 159

Nancy with a parrot obsession? Now I'm wondering if she's an Arthur Ransome fan, and if so what her (or your) thoughts on the Latin storyline in Missee Lee are. (Putting aside the 1930s attitudes towards Chinese people in the book.)

Which inspires another question I'll ask as its own thread.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

It's been too long since I read Missee Lee... Thank you, you have just put another book on my already too long reading list. :)

September 9, 2019

https://www.duolingo.com/profile/jairapetyan

Thanks Trofaste for your generous offer to clarify what you can.

Here's my question: Can GA's private message each other on Slack? I have only been able to PM one moderator/contributor, because I already have her as a contact. I have tried searching for others with no success. Perhaps my Slack is only for educators? I find it very uninteresting, really so far it's been a waste of time. I would like to PM contributors and moderators, often I have important insights, but I cannot just post them in the regular forums. Thanks again. I appreciate it. And you can pm or email me the answer if it's not a good idea to write it here.

September 8, 2019

https://www.duolingo.com/profile/Trofaste

There's no direct contact between the GA branches as far as I know.

September 8, 2019

https://www.duolingo.com/profile/skelkingur

When performing A/B tests on new trees (i.e. the Norwegian Tree 3 vs 4) what exactly are Duolingo testing for? What does it mean for a tree to perform "better" than another?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

Duo gathers different metrics from the two trees and then compares them. If those metrics improve or break even, the tree will pass. If there's a slight downward change they might do a deeper assessment of the tree to see if it's really better at the things the metrics don't show despite that and make a decision based on that, but if there's a significant decrease, the tree will definitely fail. I'm not sure how much I'm at liberty to go into details on that, but user retention is important (are learners sticking around?).

September 9, 2019

https://www.duolingo.com/profile/McKennaAW

Do you know if health is going to come to the website, or just the app?

September 7, 2019

https://www.duolingo.com/profile/Trofaste

No idea. We have nothing to do with health, that's something staff does.

September 8, 2019

https://www.duolingo.com/profile/jairapetyan

I have read an overwhelming number of complaints from Android users about health. It doesn't seem like solid instructional theory, and with experts now on the staff, I predict that it won't come to the website; on the contrary, it will be deleted from the Android app. This is just my prediction, I have no real knowledge of what is brewing.

September 9, 2019

https://www.duolingo.com/profile/jrikhal

Hi jairapetyan,

just about it coming or not to the website: Duo always announced they never planned to bring it to the website (independently of experts being at Duo or not, which, AFAIK, was already the case at the time Health was developed).
Source:

Health and Gems are only available on iOS for now. We will be testing these mechanics on Android in the future. We do not currently have plans to bring this to Web.

from Support Pages' article "What is Health?", which hasn't been updated for very long. The last sentence is there since the beginning: as you can see even before Health arrived to Android, Duo's "plan" was already to not bring it to web.

N.B.: The bolding in the quotation is mine, not in the original text.


P.S.: Is it still the plan to not bring it to the web? Is this Support Page still up-to-date despite not being updated for at least 1 year?
I'd guess so but, like Trofaste, I don't know.

September 9, 2019

https://www.duolingo.com/profile/McKennaAW

Ok. Thanks for all of you hard work! :)

September 9, 2019

https://www.duolingo.com/profile/ChevyBarnes05

Why do my contributions never get accepted? For Afrikaans, I have sent in soo many, and so has Dessert-Rose. Before Finnish was added, I had sent in a few forms, and so had Dessert-Rose. AFRIKAANS WOULD HAVE HAD CONTRIBUTORS, but instead, Finnish (at the time remained without contributors for a week or so) was added.

Also, how are skills, words and sentences added? If Afrikaans IS finally accepted, I'd like to know how this all works :D

September 7, 2019

https://www.duolingo.com/profile/Trofaste

First I'll mention that sending multiple applications doesn't help, in fact the only thing it can do is slow down the process by making staff sort through even more applications.

Applications aren't the only or even the first criteria for adding a course. Staff also looks at the potential benefit of the course, the cost of adding and supporting the course, and other factors. They can't add every course at once, so they have to prioritise somehow. If Afrikaans had been added and not Finnish, many people would have been upset about that too. There's a discussion here about how staff decides what courses to add next. :)

Finnish didn't have contributors for a week or so because staff had to sort through the many applications and choose who to invite not because there weren't any applications. This happens with most new courses. :)

Also, if/when Afrikaans is added, there have been many other applications, so you may not be chosen. But while you wait, you can improve your chances by being the kind of user that will look appealing to whoever is making thise decisions. :)

How are skills, words, and sentences added? Oh boy. That's a broad question!

We have a tree page that looks kind of like the tree page learners see, with the list of skills. On unlocked trees (trees still in development that learners don't have access to) there's a button we can click to add a skill. Then we have to put it where we want it in the tree, and give it a name and choose the icon from a list.

Then we can go into the skill and add lessons, and add words to the lessons. How to create a word depends on what language the course is teaching, but basically you write the word and then enter some basic grammar about it (verb, 3rd person, singular, past tense, masculine, etc.). That creates a unique lexeme which the system now knows is being taught here and can be used after this point but not before.

To write a sentence you go to a word and click a button which opens a menu. You type the first word, stop, wait for a dropdown to appear, select the right lexeme, type a space and the next word, wait, repeat, until the sentence is done. Oh, and you need to hit enter after every punctuation mark or it won't register. Then another enter at the end to finish the sentence. This can result in some interesting typing patterns in normal writing after writing a bunch of sentences! Then you have to enter the best translation of the sentence, and add alternative translations. The system takes that best translation and uses it for the reverse exercise (translation back into the target language). The best translation for that has to be the original sentence, but it also needs accepted translations added. Then you need to check the hints for both directions, and if it's a language with recorded audio, someone needs to record it. If it's a TTS language it takes a few days for the file to be generated.

September 8, 2019

https://www.duolingo.com/profile/ChevyBarnes05

Oh wow, that's a lot of responsibility. I get what you mean about the forms and everything. I haven't actually submitted for a while. I'd like to help with the course, but first, you have to think about the benefit.

In the mean time, I'd like to try and make myself into the type of user to be accepted into a course. Thanks! This is by far the most informative post I've ever read!

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

One thing you could do, if you haven't yet, is create some Afrikaans lessons in the forums. I think that's what one of the Finnish contributors did before the course entered the incubator, and they got accepted as a contributor. You can have a look here for examples of user created lessons in the forums: https://duolingo.fandom.com/wiki/User-created_lessons

September 9, 2019

https://www.duolingo.com/profile/ngraner42

Does the new spaced repetition system use an increasing period of time between reviews, like Anki. If so what is the increase factor. Anki defaults to 2.5, does your research indicate that a different factor is preferable. Thanks.

September 7, 2019

https://www.duolingo.com/profile/Trofaste

I don't know the answer to that, volunteers like me only work in creating the course content. The spaced repetition is handled by staff.

I have a vague memory that they may have had a blog post about their spaced repetition system a long time ago, but I don't remember for sure, and I don't know how much lf the information would still be current.

September 8, 2019

https://www.duolingo.com/profile/Little_Tatws

What's your favourite part about being a contributor? What's your least favourite?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

My favourite part? Being able to have a positive impact on so many people. The random conversations that sometimes happen are pretty fun too! My least favourite part would be wading through the particularly abusive reports. It gets rather tiresome being informed you don't speak a language because there was a mistake in a sentence or one variation got left out, or being told what a horrible person you are for writing such a sentence (on sentences ranging from "The girls eat butter." to "The shark is eating a dolphin." to "Elephants are large."), or having profanity screamed at you in all caps for who knows what reason, or the extremely impolite instructions on what we should have written, which are usually wrong.

September 9, 2019

https://www.duolingo.com/profile/Larisa_L

I used to get upset about those ugly messages too, but now I've reached nirvana, and don't care. I find it now more upsetting when the Duo staff makes some "improvements" which end up making life harder for us.

And my most favourite thing about being a contributor is a chance to work with really awesome people like Trofaste :)

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I would say awesome people like Larisa, and a few others I could name, several of whom have commented in this thread. :)

September 9, 2019

https://www.duolingo.com/profile/ehartz

The amount of users who very confidently (and often rudely) assert that they are right about something when they are completely wrong often borders the line between amusing, exasperating, and infuriating in my experience. To use an example based off of the screenshot Trofaste shared above, it isn't totally out of the norm for us to get freewrite reports that say, "ARE YOU STUPID? THE CORRECT ANSWER IS 'IT IS A WOMEN'!"

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I would even say it is the norm to get at least one of those per sentence.

September 9, 2019

https://www.duolingo.com/profile/Joshorst

Are all sentences in a lesson explicitly added as such, or is some level of computer aided generation involved?

So, for instance, I get to translate 'the red dress', the next guy gets 'the red skirt'.

September 8, 2019

https://www.duolingo.com/profile/Trofaste

They're all manually added and all translations are manually added. There's no computer generation involved at all. So if you have "the red dress" and the next guy has "the red skirt", it's because we manually created both.

September 9, 2019

https://www.duolingo.com/profile/ILoveYou3000.

Which duo course is your favorite? (fun or practicality)

September 8, 2019

https://www.duolingo.com/profile/Trofaste

That's a dangerous question to answer when contributors from different teams are watching... ;-) I really like the Norwegian course, because the contributors are awesome and have so many wonderful sentences. They have an unfair advantage in that they're teaching a language with fairly simple grammar so they can add all kinds of fun vocabulary.

September 9, 2019

https://www.duolingo.com/profile/Deliciae

I just want to make it clear that I did not bribe Trofaste to say any of this. All I did was threaten to feed her to the birds.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

You forgot to tell me you weren't bribing me, I thought... Oh wait, I probably shouldn't say that on a public forum, I don't want to ruin our reputations.

September 9, 2019

https://www.duolingo.com/profile/endless_sleeper

I do remember something about brunost...?

Oops, shouldn't have mentioned that.

Trofaste is right though. Norwegian is awesome! ^_^

September 9, 2019

https://www.duolingo.com/profile/Trofaste

SHHHHHH!

September 9, 2019

https://www.duolingo.com/profile/MonikaEggers

Can contributors see the other trees in contributor mode, too? E.g. could someone contributing to “German to Danish” rely on the existing tree “English to Danish”, at least by copy and paste (or it seems by typing off word by word as selecting lexemes is required after each word)? Using the existing Danish sentences would be helpful in this example, even if the tree could not be exactly the same (e.g. because English you = German du/ihr/Sie = Danish du/I and similar issues).

September 9, 2019

https://www.duolingo.com/profile/Trofaste

We can see other trees, but only read-only, we can't edit them, and there are certain things only the contributors to that course can see. When creating a new tree we can either start from scratch or use an existing tree as a template (which can then be modified), which would be a viable option in your example. And we can always seek inspiration in other trees, for teaching ideas and also for creative sentences when we get stuck.

September 9, 2019

https://www.duolingo.com/profile/Mereade

Hi, thanks for this opportunity to ask! How much time do you spend creating the course weekly? What is the norm? And how much work do you get done in that time? It is a bit difficult to imagine, how fast or slow does the work go, for the individual contributors.

September 8, 2019

https://www.duolingo.com/profile/Trofaste

It varies somewhat from week to week, but it's generally somewhere from 10-25 hours. Probably mostly in the lower end of that range for me. That also varies from contributor to contributor.

The work I do is hard to quantify as I work in different roles on different courses and I also spend much of my time in behind the scenes work, helping other teams with their technical issues, bug hunting (the Incubator is riddled with bugs), and other things like that. But if I were to spend a solid hour clearing reports... It would depend on the course and the skill. Some take longer than others, for example, Czech is more complicated than French, and the farther down the tree you go the more complicated the sentences get so the more you have to check in each report. Smaller languages also have fewer reports per sentence which means changing sentences more often, which means waiting for the Incubator to load it (the Incub is painfully slow)... On an average skill in the French course I could probably clear several thousand reports in an hour, quite possibly without finding a single correct one. On average in the Czech course, I would be lucky to do several hundred in the same time.

Adding sentences is even harder to generalise. It would depend on how quickly I could come up with the sentence (not always as easy as it sounds!), the complexity of the languages involved (relevant for the translations), how many translations there are to add (a language with flexible word order has a lot more to add than one that doesn't), etc. If I'm adding full translations and hints myself (which I usually try to do, then someone checks my work), then with "simple" sentences like "I am a man." and "You are a woman." I could do 10-15 an hour, maybe 20 if the languages involved don't have many alternate translations. Once you get into more complicated sentences with a lot of translations, 5 an hour is doing well.

September 9, 2019

https://www.duolingo.com/profile/hanspersson

Would it be possible to share something about how the correct answers for a sentence are arranged? Is this simply a list of correct answers with minor variations or can you build something like a tree where multiple variations of one word can be entered inside the same sentence (like a regular expression in programming, if you happen to know what that is), i.e. "(I am|I'm) (happy|glad)" to specify four variations in one.

September 9, 2019

https://www.duolingo.com/profile/jrikhal

Hi hanspersson,

This answer to another question might interest you.

September 9, 2019

https://www.duolingo.com/profile/Mereade

Thanks for a great answer! I can imagine it much more clearly now.

September 9, 2019

https://www.duolingo.com/profile/Zia177448

Is there a lot of communication between contributors and staff or are there mostly seperate circles?

Do you often get told about changes that will be made to duolingo ahead of time? Do you often have to adjust things in the courses because of these changes?

Do you contributors have a lot of in-jokes?

In a contributor team, how is the work load distributed? Is there usually one person who sets the rough benchmarks of when to introduce what grammar skill etc and others who prefer the work of coming up with the sentences? Is there a lot of discussion about how to do things?

Do you have the chance of suggesting things to the staff/developers that would improve the course on a technical level?

When looking through the reported comments, do you have the chance of rejecting some wrongly suggested alternative once and for all (so that any reports with that exact suggestion automatically get rejected)? I'd also love to see a screenshot of how the interface for reported sentences looks, if you are allowed to share that. (Eta: Has already been answered) (I'd generally love peeks into the Incubator, I love seeing what goes on behind the scenes.)

Does something special have to happen for a contributor team to decide that they should start working on an expansion of a tree or is the next expansion of a tree something that is usually at the back of the mind and something that will be worked on whenever there's time for it?

If you work on both a tree and its reverse tree, which one do you personally prefer?

(Sorry that I have so many questions (and please just pick whichever ones you want to answer), I'm just always really interested in what goes on behind the scenes.)

September 8, 2019

https://www.duolingo.com/profile/Trofaste

Is there a lot of communication between contributors and staff or are there mostly seperate circles?

There's a specific team of staff responsible for the Incubator and courses, we have two main contacts in that team but we see several of them from time to time.

Do you often get told about changes that will be made to duolingo ahead of time? Do you often have to adjust things in the courses because of these changes?

No, we don't. We usually know things no sooner than users, in fact, we frequently get our information from users. Most often changes only impact the way the content we create is presented rather than what content we create or how we create it. The changes that affect us are usually behind-the-scenes changes that users don't actually see.

Do you contributors have a lot of in-jokes?

It's possible... More sentences than users will ever know might perhaps come from them... We also might have silly debates from time to time, like about what coffee actually is and what should be put in it... And we might occasionally share particularly amusing comments or reports... But you don't know any of this.

In a contributor team, how is the work load distributed? Is there usually one person who sets the rough benchmarks of when to introduce what grammar skill etc and others who prefer the work of coming up with the sentences? Is there a lot of discussion about how to do things?

That varies widely from team to team. Generally there's one or two people more or less in charge, especially on older courses, where the contributors who've been around the longest are typically the team leads, but there's generally a lot of discussion. The work on maintaining an existing course is all much the same. There might be one person who handles adding/improving Tips & Notes, but most of the work is deleting bad sentences, adding new ones, and clearing reports. When working on a new tree there will usually be some division of labour, but how it's done will vary.

Do you have the chance of suggesting things to the staff/developers that would improve the course on a technical level?

Well... We can suggest things all we like. Getting them done is another matter...

When looking through the reported comments, do you have the chance of rejecting some wrongly suggested alternative once and for all (so that any reports with that exact suggestion automatically get rejected)?

Sort of. We can write a message on a report, which used to be shown to users. It isn't anymore, but it does stop the report showing up in the list for us again. It's still counted in numbers like the reports per 100 users, however.

Does something special have to happen for a contributor team to decide that they should start working on an expansion of a tree or is the next expansion of a tree something that is usually at the back of the mind and something that will be worked on whenever there's time for it?

It's always in the back of the mind, and usually thought about even while the previous tree is still in the works. It's usually started when the team feels the current tree is stable enough and they get bored doing the mundane work of maintenance (new trees are the fun part!).

If you work on both a tree and its reverse tree, which one do you personally prefer?

Well, if a course has a reverse course, then it's the reverse course of its reverse, so there's not much between them in that respect. Personally I prefer courses that don't have a reverse tree, because the shared database between reverse courses is a pain in the neck.

(I'd generally love peeks into the Incubator, I love seeing what goes on behind the scenes.)

I saved this for last because it will be long. :)

I already shared screenshots of reports, but here are a few others as well.

tree view

This is a small snippet of what the tree looks like in the Incubator. The skills are gold because all the words in them have 3+ sentences. The top number is a meaningless number that we don't use for anything, the lower number is the number of reports in that skill. It stops at 5000+, even though those skills have between 20k and 80k reports each.

inside a skill

This is inside a skill. "All reports" opens a column that shows all the sentences in the skill sorted by total reports of all types on the sentence. The left column is the list of lexemes taught in the skill, divided into lessons.

sentence exercises

The sentence exercises created for teaching a word, in this case "am" in English. The ones with a green checkmark have been translated and are live. The ones with the grey checkmark haven't been translated and aren't shown to learners. The red percentage is a measure of how difficult learners find the exercises with that sentence.

Forward exercise

Example of a "forward exercise" in Czech<-English. The blue sentence is the Czech sentence to be translated into English. The line with the blue star is a "Best Translation" (BT). This is the one shown as a correction when you make a mistake. It's also used as the English sentence for translating back into Czech. The line without a blue star is "Accepted Translations" (ATs). These are translations that are accepted but aren't (supposed to be) shown to users unless they give that answer.
The words in brackets, like "[The/A]", are in "compact form". This lets us enter multiple translations that are identical except for one word without needing to write them all out separately. For example, if we accept "[This/That] is a dog." then both "This is a dog." and "That is a dog." will be accepted. It works if you have multiple sets of compact form too - "[This/That] is [an apartment/a flat]." will accept "This is an apartment." "That is an apartment." "This is a flat." and "That is a flat." You have to be careful with it though, if we wrote that as "[This/That] is an [apartment/flat]." then we'd have sentences with "an flat" rather than "a flat". That kind of thing is where many of the errors creep in, because it can be easy to miss something like that!

Reverse exercise

This is the "reverse exercise" (translation from the source language back into the target language) for the same sentence. As you can see, the blue words are the English sentence, the same one as the BT in the forward exercise. Again, the line with the blue star is the BT. This has to be the same as the original Czech sentence. The other lines are ATs, as above. As you can see, there can be a lot of translations (this sentence has 32, which is actually a small number for Czech). Compact form reduces that to only 4 lines in this sentence, but care has to be taken that you aren't accidentally accepting something you shouldn't be, or missing something you should have. When you get to sentences with thousands of translations, that can be tricky!

September 9, 2019

https://www.duolingo.com/profile/MonikaEggers

As a contributor, do you have influence over words/sentences labeled as „you have a typo“ (vs. marking other replies with also just one letter off as wrong) or is that done wholly by algorithms without human intervention?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

No, we don't, typos are done by algorithm. Usually if it's one letter off it's accepted as a typo unless that makes another word, when it will generally be marked wrong (as then the system can't know if it was just a typo or actually using the wrong word).

September 9, 2019

https://www.duolingo.com/profile/hanspersson

If I remember correctly, one sentence I got accepted ("with a typo") in Latin the other day had five incorrect letters in three (out of four) different words.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

If it's two letters switched around (eg. recieve instead of receive) that will generally be accepted, and you can have typos in as many words as you like as long as they're all recognised as typos. There are also a few exceptions to the rule which allow a couple letters to be off.

September 9, 2019

https://www.duolingo.com/profile/Linda7Italian

Trofaste. Love your post. Shine up that bodyarmour and prepare for "Lgs and Dvs;-) Cheers and tks in anticipo.

September 7, 2019

https://www.duolingo.com/profile/Trofaste

Thank you Linda! I must confess I stole borrowed most of the post from someone else's AMA 4 years ago and just modified it.

September 8, 2019

https://www.duolingo.com/profile/Joshorst

Had to mention Dvs again?

September 8, 2019

https://www.duolingo.com/profile/ISpeakAlien

What are your responses to the questions that I ask in my incubator survey at https://forms.gle/DhAt1J2mhrUnV3DG6 ? You can also look at the survey to see the options provided because some of the questions aren't written very well.

The most relevant questions (which have been edited a little) are:

How does Duolingo pick what courses to add next?

Who is qualified to contribute to a course?

How does Duolingo pick which applications to accept?

When are applications considered (soon after they are made, or when Duolingo decides to add a language)?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

How does Duolingo pick what courses to add next?

They use a variety of criteria. They discussed it in more detail here.

Who is qualified to contribute to a course?

If you speak both languages at a native level (even if you aren't native in both), you know the grammar of both thoroughly, you're willing to commit significant time to it and stick to it, to do extremely boring and frustrating tasks for hours on end, and to receive no reward for doing it, you're probably qualified. You might also be interested in the comment I wrote here about what I'm looking for in an application.

How does Duolingo pick which applications to accept?

Duolingo themselves usually only choose the contributors when starting a new course, and I don't know the exact process, but I assume they base their choices on recommendations from trusted people, quality of language in the applications, and activity on Duo. On existing courses, the contributors choose who to invite, and generally base it on the text of the application and forum activity. The comment I linked above is relevant here too. :)

When are applications considered (soon after they are made, or when Duolingo decides to add a language)?

That depends. If they're for a not-yet-existing course, then it will be when Duolingo decides to add a language. For existing courses, it varies. Some teams only look at the applications when they're actually looking for a new contributor. Some stay on top of them, rejecting the ones that aren't up to scratch as they come in and saving good ones for when they want to look for a new contributor (this is a lot more practical on smaller courses which receive far fewer applications). Some do a mix of the two, occasionally going through the applications clearing the bad ones.

Should it be possible to create a course without approval from Duolingo?

No. Course creation is not simply a matter of adding the course to the Incubator, adding contributors, and leaving it to run. There are ongoing costs to Duo in supporting the course, and they have to assess those costs in determining whether they have the resources to add a particular course at a particular time.

Should it be possible to edit an already-existing course without approval from Duolingo?

No. This would be a recipe for many errors in the courses. We receive many, many incorrect error reports. If anyone could edit the course, those would be done as edits instead, which would mean errors in the courses that someone would have to fix, and end up as a perpetual edit war between people who think they know the language (even though they're just learning it) and people who actually do know the language.

Should all users be able to do courses as soon as they enter the Incubator?

Well, when a course enters the Incubator, there is no course to do... You might as well be able to do courses before they enter the Incubator. Courses are released into beta once they're user ready, it would be pointless to allow all users to take them before that point.

September 9, 2019

https://www.duolingo.com/profile/ISpeakAlien

Wouldn't it make way more sense for Duolingo to pick the courses based on which ones receive applications? I saw on the link that you provided "But sometimes the stars align and we’re able to create courses like High Valyrian." That's a very strange response...

Is it true that people are not contacted when their applications are rejected? That is what I've heard in the past, but I'm not certain.

My questions about "without approval" both mean "without approval for initial creation". The idea is that the new/modified course would have to be approved before being made public, but Duolingo would not have to agree before the course can start to be made.

I have one last question. Could you please provide a sample course application that was accepted, or if that's not possible, could you provide a sample course application that you would accept?

Thank you for this AMA.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

No, it wouldn't. Duo gets a lot of applications for a lot of courses and they don't have the resources to support them all right now. They have to prioritise somehow.

If we positively reject an application, the user will get an email informing them. They will not get it if the application is kept as a possible future option.

Duolingo has to agree because a) contributors have access to sensitive information and have to be vetted, and b) they have costs supporting the course.

Well, one of my applications was a translated piece of an old essay... But it was just filling in the space so I could send it so I could be added to the course, because it had already been decided that I was joining that team. It's not generally recommended. :)

I'll take a hypothetical course, Klingon for High Valyrian speakers. Pretend that the application has been written in fluent Klingon and translated into flawless High Valyrian.

"Hi! I'm KlingonFan and I'm interested in helping create this course. I'm a native speaker of High Valyrian and fluent in Klingon. I've studied Klingon for 20 years, and have a certificate in teaching Klingon as a second language from the University of Qo'noS. I also have a degree in linguistics. I would like to help create this course because I believe there are many High Valyrian speakers who would benefit from learning Klingon. There are currently very few resources to help them, and I believe a Duolingo course would have immense value to this community. I am able to consistently contribute 10 to 15 hours a week to working on the course.

I've created some resources on the forums, which you can see here (link). I've also been a regular Duolingo user for a year, and am at a high level in the Spanish for High Valyrian speakers course, so I am familiar with how Duolingo works for users.

Thank you for considering my application. :) I hope to be able to work on the course in the Incubator soon!"

This would be a promising application that would be worth looking at. Of course, the application is only one thing, and I would look more at their forum activity (and I've linked elsewhere to a comment where I explained what I'm looking for), but this application would make me consider it worth it and likely not a waste of time.

September 9, 2019

https://www.duolingo.com/profile/endless_sleeper

Do you have a pet owl?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Yes. He lives on 20 some browser tabs.

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Thanks for doing this Trofaste! I have a few questions :-)

  • What courses have you contributed to and which are you currently contributing on?
  • As a lead on from the previous question, which courses are you expanding on (making a new course for), and which courses are you working on dealing with bad sentences and reports?
  • You seem to contribute to a lot of courses. How many languages are you a fluent speaker in and how many courses are you working on but aren't fluent in? For example I think I saw you're a contributor to the Navajo course? Are you fluent in Navajo or how did you become a contributor?
  • How many lingots do you have? I'm on 2650 :-)

Thanks for answering any of my questions and thanks again for doing this AMA! :-)

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Since you asked... I currently have contributor status on French<->English, Polish<-English, Czech<->English, French<-Spanish, German<-English, Hawaiian<-English, Navajo<-English, English<-Telugu, Latin<-English, and Romanian<->English. I think that's all. It's also extremely atypical, don't think all contributors are anything close to that (most are on one, or often two if both courses in a language pair exist).

Czech<-English is slowly working on a new tree, as is Polish<-English. Others might be working towards it, or may have one already in testing/just finished testing, but if they are it's slow and will be a while (as it will be for the two I mentioned). My report clearing efforts are mainly on English<-French, Polish<-English, and Czech<-English, and right now on Latin<-English to help with the initial report flood on entering beta.

How many languages am I fluent in? One. English. I'm also advanced in French. It's unusual for a contributor, but I'm also kind of an unusual contributor, with much of my focus actually being on work that isn't exactly part of any course, such as mentoring new contributors, helping other contribs with technical issues, and bug hunting. So many of the courses I'm on I'm on as a mentor to teach the team how to use the Incubator (Navajo is one of those). Others I'm on for a specific task (eg. PL<-EN I'm primarily on for working on the English side, Czech<-EN I'm on mostly as an advisor for the new tree, because while I'm not fluent, I understand how the language works pretty well, I know what learners struggle with from the learners' perspective, and I know the Incubator pretty thoroughly, etc.)

1801 lingots as of the last time I reloaded the page, but it varies a lot, because I get given a lot of lingots and I give a lot of lingots to helpful, sensible comments. :)

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Woah nice! Thanks for answering my questions! Hope it's ok to ask a few more? :-)

  • How long have you been using Duolingo?
  • What was the first course that you started contributing to and how did you become a prominent contributor?
  • What got you interested in languages? Did you study linguistics?
  • How much time do you spend roughly contributing to Duolingo?
  • How did you start mentoring new contributors?
  • How long have you been a duolingo contributor for? When did first start?

Thanks again for doing this, and for answering my questions! I'm glad we have people like you (or a person like you if you are unique) helping with the Duolingo courses! Do you only use the duolingo forums or are you active on any other internet platforms?

Nice! Thanks for the information about the Czech and Polish courses. Looking forward to seeing the new trees one day!

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I've been using Duolingo since... spring/summer of 2016, I think it was?

My first course was Polish<-English. I've got where I am now by being regularly active for a long time, being good with technology, and constantly bugging the Incubator's brain (you know who you are ;-) )!

I'm not sure when it first started... In school languages were an excuse to try to avoid work in English, which I absolutely hated (except Latin, which I also hated), but I didn't like them that much and forgot most of what I learned. It probably comes from knowing people from different places who speak different languages since very early childhood.

How much time I spend varies from week to week but on average probably somewhere from 10-25 hours a week, most often in the lower end of that range.

The Incubator has always been a place where contributors ask other contributors for help when they get stuck or have a technical issue. I'm good with technology so I learned a lot about how the Incubator works quickly and started jumping in to answer questions where I could. And I've been learning more about the Incub for a long time now. So that's how it really started.

I've been a contributor for over two years now, I first joined the Incubator on 24 July, 2017. :)

I'm not unique, there are many contributors far more awesome than I am. I won't list them because I would be sure to miss people who should be added, but you know who you are.

What other internet platforms I may or may not use is a secret. :p

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Woah nice! Thanks for answering my questions and doing this AMA! :D

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Also, do you know anything about the Englsh for Danish course that entered the incubator around April/May 2019? It got removed a few days later. I saw a comment saying that no course has ever been remvoed from the incubator apart from an April's Fool course, and one that was added by mistake? Was that the English for Danish speakers course?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

It was.

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Thanks! :-)

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Also, another question I have is what is the difference in the incubator between those that have a orange ring and those that don't, for example in the Latin course? https://incubator.duolingo.com/courses/la/en/status

And thanks again for doing the AMA! I'm finding it interesting learning new things so I truly appreciate it!

One other question I have is which courses are most in need of alpha testers/helpers at the moment? For example I'm not particularly proficient in any language apart from English and a little Russian, but I would like to help with some courses in any way I can as I do have a lot of free time. Thanks again for answering my questions and if you have any information or insight into applying as a alpha tester, even without having a lot of knowledge of the language being taught, I'd truly appreciate it! :-)

Also what have you been up to today apart from answering questions on this AMA? :-)

September 9, 2019

https://www.duolingo.com/profile/daKanga
Mod
  • 302

If Trofaste and others will excuse me hopping in regarding this:

what is the difference in the incubator between those that have a orange ring and those that don't, for example in the Latin course?

Generally to read about what the different rings mean, check out: [GUIDE] Avatar rings and Global Ambassadors

However your question is more specific than that, and concerns the difference between Green and Gold contributors.

edited: In Phase 1 the only real difference is actually that only golds can lock the tree and invite new contribs. Even greens can add words.

which courses are most in need of alpha testers/helpers at the moment?

It is best to watch the forums for discussion posts that contributors may put up, and that they may also request to be stickied in their individual forums. And/or watch the notices in the incubator, that some teams use to announce course progress.

If you go to GUIDE Find a Discussion forum/board/list, it will give you links through to the various incubator status boards, via the emblem.

Also to keep your eye out for the weekly Weekly Incubator Update WIS (WIU)

Also what have you been up to today apart from answering questions on this AMA?

Of course - I feel you are asking Trofaste this question. So I will not interject on this one.

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Thanks for answering daKanga! I appreciate it! I always wondered what the difference was between the gold contributors, and those that don't have a ring so now I know! :D

September 9, 2019

https://www.duolingo.com/profile/Trofaste

As daKanga said, watch the forums and the Incubator updates for alpha testing positions. If a team needs alpha testers, they'll let users know, and say what they're looking for in an alpha tester application. :)

Today has been real life things and some contributor work (mostly of a sort of administrative kind). :)

September 9, 2019

https://www.duolingo.com/profile/Ccf-Uk_2018

Sweet! Thanks Trofaste! :D

September 9, 2019

https://www.duolingo.com/profile/Songve

Is it true we are not allowed to use the term to describe action taken against a user that enables them to do course work and make posts but no one sees them?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

I'm not free to discuss the tools we may or may not have for keeping the forum clean.

September 9, 2019

https://www.duolingo.com/profile/garpike

This is easily experimentally tested: shadowbanning, shadowban, shadowbanned. If this post mysteriously disappears, then you'll have your answer.

September 9, 2019

https://www.duolingo.com/profile/E.T.Gregor

Unless I'm misreading something, it sounds like there are a lot of things you are not allowed to answer. Is there some strict privacy policy for the incubator? Why is sharing screen shots such an issue?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

There is, and we have to sign a code of conduct when we join which includes agreeing not to share confidential information about the Incubator and the inner workings of Duo. Everything I've shared on this thread has been given the green light by staff.

September 9, 2019

https://www.duolingo.com/profile/E.T.Gregor

Interesting. I wouldn't have thought of the incubator as something to keep secret.

September 9, 2019

https://www.duolingo.com/profile/davidcwalls

I'm not a contrib, so I know nothing about the incubator or Duolingo's internal development process, etc, but I have been developing software for 3 decades, and I can offer some observations from that perspective: users subconsciously build mental models based on the way software looks and works, and those models influence the way they think about the software and the organization that provides it. In a sense, every item on the screen is part of an implicit contract with the user, e.g. the "Post" button I see tells me that I will be able click the button and make this comment visible to the community. And once you have offered some functionality (even accidentally), users will feel betrayed if you take it away.

When you develop a tool for internal use, you don't want to have to put as effort into thinking through every pixel on the screen and what a user might make of it - you've got internal people who need to perform some task, so you want to get them the tool ASAP and worry about refining the UI later. Also, you don't want the general population of users to think (based on seeing a screenshot of a tool), "doing this thing must be easy because I saw a button for it, so if the internal people aren't doing it they must be stupid or lazy, etc." when really the button doesn't work, or using it requires 100 other prerequisites...

September 9, 2019

https://www.duolingo.com/profile/Trofaste

There's also the factor that we could reveal information about the internal workings of Duo that they don't want widely known. They are a business, after all, and businesses aren't usually fond of sharing the details of how they create their product with the entire world. :) It could also cause problems for testing, press embargoes, and official announcements.

September 9, 2019

https://www.duolingo.com/profile/piguy3

Does the Latin course plan to accept every conceivable word order, or only those that would have been used with a certain minimum frequency?

Is the Navajo course in something of a "holding pattern" until another proficient speaker can be recruited/re-recruited to contribute?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I believe the rule being followed in the Latin course is to accept any word order that can be reasonably justified. So no "non" after the verb, and prepositional phrases like "in urbe" can't be separated, but as long as you keep the units of meaning "[studet] [Livia] [in urbe]" together you can do pretty much what you like with them.

I believe that's the case with the Navajo course, yes.

September 9, 2019

https://www.duolingo.com/profile/Jenna517810

Which courses did you contribute to and how did you originally learn those languages?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

I currently have contributor status on French<->English, Polish<-English, Czech<->English, French<-Spanish, German<-English, Hawaiian<-English, Navajo<-English, English<-Telugu, Latin<-English, and Romanian<->English. I actually only speak English (my native language) and French at an advanced level, but I'm on those teams in several different roles. I initially learned French at school, but forgot most of it, and most of my study was done on Duo. Originally with the French<->English courses, then in Immersion when that was a thing, following Sitesurf around the forums reading every single comment she wrote (literally), and then moving beyond that to reading books and other things.

September 9, 2019

https://www.duolingo.com/profile/kaet
  • 159

Do you have any favourite books featuring language learning as a major plot point?

September 9, 2019

https://www.duolingo.com/profile/Trofaste

The Story of the Trapp Family Singers by Maria Augusta Trapp is a favourite, and while I wouldn't say language learning is a major plot point, it definitely figures.

September 9, 2019

https://www.duolingo.com/profile/kaet
  • 159

I'll look for that, thanks!

September 9, 2019

https://www.duolingo.com/profile/daKanga
Mod
  • 302

The hills are alive with the sound of music ???

Duo1

edited: you have now made me look for real information.
And as is common, I think the book of their real story sounds a great book, that I would enjoy looking into. i.e. https://www.archives.gov/publications/prologue/2005/winter/von-trapps.html

As always, thanks Trofaste !

September 9, 2019

https://www.duolingo.com/profile/carbsrule

How do you like to eat potatoes?


Manĝu terpomojn kaj feliĉiĝu!

September 8, 2019

https://www.duolingo.com/profile/Trofaste

Pretty much any way, but I'll pass on the baked potato served with ice cream and cheese.

September 9, 2019

https://www.duolingo.com/profile/ISpeakAlien

What's the point of this "manĝu terpomojn kaj feliĉiĝu" thing?

Kio estas la kialo de tiu "manĝu terpomojn kaj feliĉiĝu"-afero? Mi ne komprenas ĝin.

September 8, 2019

https://www.duolingo.com/profile/carbsrule

Ĉu vi ne legis aŭ ne ŝatis mian antaŭan respondon?

Nu, ekde la pratempo, en la forumoj de la interreto, oni havas sian propran devizon. Tio estas unu el la miaj. La alia estas "mi kredas je la manĝado de terpomoj". Mi ekpensis pri l' afero en la internacia jaro de la terpomo, 2008.


Manĝu terpomojn kaj feliĉiĝu!

September 8, 2019

https://www.duolingo.com/profile/ISpeakAlien

Mi ne legis ĝin, sed mi serĉos ĝin post kiam mi skribas ĉi tiun mesaĝon.

How do you put the message at the end of every post? Do you add it manually or is there a setting somewhere?

September 9, 2019

https://www.duolingo.com/profile/waveclaw

Do course contributors influence the Quiz for each language available on the website edition or are those selections generated or curated by different team?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

I'm not sure which quiz you're talking about, but it's probably generated from the content we create.

September 9, 2019

https://www.duolingo.com/profile/E.T.Gregor

This seems to be a test or a feature only for some platforms, but there are now "checkpoint challenges" (where the test-outs used to be) that you have to pass in order to access the next section of the tree. From the content, I would also guess that they are auto-generated.

September 9, 2019

https://www.duolingo.com/profile/Trofaste

Ah, those. Yes, those are auto-generated using the content we've created in the course.

September 9, 2019

https://www.duolingo.com/profile/ColinJParry

What's your favorite color?

September 8, 2019

https://www.duolingo.com/profile/Trofaste

Yellow. It's my substitute for sunshine!

And that's my last question for the moment, I'm afraid the rest of will have to wait till after I've had some sleep!

September 8, 2019

https://www.duolingo.com/profile/jairapetyan

I have read that looking at yellow or orange can improve your mood!

September 9, 2019
Learn a language in just 5 minutes a day. For free.