Suggestion: Common mistakes
In the discussion area for many questions, people often post what they think should be accepted answers. Sometimes these really are correct; other times they are common mistakes. Other users often provide valuable feedback, but I think this can be incentivized into a better feature.
Duolingo could count the incorrect responses to questions (using a string distance algorithm to make sure slight variations are counted together). When an incorrect response reaches a particular threshold, it goes on the list of common mistakes.
Users can be encouraged to write explanations for these common mistakes, perhaps by providing an extra half heart. When I miss a particularly tricky question, for example, I often look up some supplementary information to find out exactly why my response was incorrect. I am imagining here that Duolingo would tell me my response was a common mistake, and encourage me to find and write an explanation for it. Then when another user enters that common mistake, they can be shown the explanation.
This may be a lot of overhead for something that is basically streamlining what already happens in the discussion for each question, but I think it would be a nice community-powered feature. A simpler alternative would be to simply provide a separate "Common mistakes" discussion area, separate from the general discussion area for each question. A separate area would encourage contribution, where users will manually point out and explain common mistakes.
Something similar is currently running as an A/B test, 50% of users should have it. :)
Explanations are written by course contributors based on common incorrect reports.
[Edit] Here's an example.
Hmm, I wonder if I'm in the test group. My question is where exactly do we get these errors, is it in the email responses, or immediately after submitting a report within Duolingo?
Also are the teams using any existing research to select the most common reports? As I said in the previous post these have already been extensively researched before, and it may help to greatly filter through the repetitive errors L2 speakers make. Apparently there are only 10 major categories of mistakes English L2 learners make, so those findings may be important.
You get the error message, like the one shown above, if you input a specific wrong answer that has a been given explanatory error message by the contributors. The error message pops up as you press enter and you lose a heart. You can still report the sentence but your report will not be seen due to the way these messages are handled (to clarify here, if you report that your answer should be correct as well it will not be seen, if you select the other options eg missing hints then it will still be seen).
The teams don't use any particular research for this, rather they have the option to add explanatory error messages to any incorrect answer that is suggest by a user. For example on the Hindi team we see a lot of users reporting that something like "I read book" should be a correct answer since Hindi does not use articles. Clearly this is wrong, so when we add an error message we say something like "You must use the article "the" or "a" here". From then on anyone who answers "I read book" will see that message.
At the moment we simply add error messages to common errors that we see. In the Hindi team we are trying to add an error message to every report we get that feels like it could be reported again at some point (so excluding spammy or nonsense reports) however the focus is to definitely have these error messages on all common mistakes that we see multiple times. I cannot speak for what the other teams are doing.
It would be interesting to incorporate research into common English mistakes here but at the moment the technology is not advanced enough to deal with that. We have to manually add every report message and we can only add a message after that particular wrong answer has been reported as "this should also be correct".
Thanks for the detailed description!
(to clarify here, if you report that your answer should be correct as well it will not be seen, if you select the other options eg missing hints then it will still be seen).
I haven't tested the system, but I think the report function should also spit out this error, to further dissuade people from sending that message, and also show the message again in case they didn't see it before.
The teams don't use any particular research for this,
I see, so you're doing it reactively rather than pro-actively. The system is still new so we'll have to wait and see, but I think that it always best to add the messages before they are submitted. For English people learning portuguese will always have a difficulty understanding that all words have a gender. There are similar attributes in all existing languages.
Anyway, I know all of this is out of your hands. I'm just pointing out my perspective.
Just one other question, how many common mistakes messages have you added on average (just an estimate will suffice) ?
You are welcome. I do often realise that as a contributor I see how things work under the hood where normal users don't.
but I think the report function should also spit out this error, to further dissuade people from sending that message, and also show the message again in case they didn't see it before.
This is a fair point. Duo is still testing this feature so I presume they're still looking for the best way to show this information. Perhaps I will suggest this to them.
but I think that it always best to add the messages before they are submitted.
Yes I agree. I hope that this feature can be vastly expanded in the future and I hope that the current tests show Duo that this is very useful and could be improved even more.
how many common mistakes messages have you added on average (just an estimate will suffice) ?
We have added quite a lot. Sorry it's hard to read but I couldn't find a way to upload this image without losing quality. As you can see a large chunk of these are from people tying to input Hindi using the English alphabet - duo is working on improving this. I don't know about the other teams.
I guess I'm not part of the test group. I think the only problem with this feature is the fact that the explanations aren't referenced. I'm not indicating that the contributors aren't reliable/competent individuals, but adults(and children) tend to contest everything unless they have proof. For example, in a recent discussion someone said a Duolingo sentence is wrong, because "their grandpa said so". Maybe referencing a grammar book or something.
This will become a more serious issue if a contributor makes a blunder and puts in a wrong explanation (to err is human after all).
This is especially important when one considers that some contributors will use grammar concepts such as " article" which don't exist in some languages (e.g. chinese). EDIT: Oops I'm not part of it, the message I got was the default Duolingo message.
Good point. We hoped to be able to put URLs in these messages linking to grammar explanation posts in the forum but this is not possible. This is because Duo (rightly) says that if we have links in the messages too many people will click the link and accidentally leave the lesson. Perhaps the tips and notes section is the best place to add these explanations but we are still thinking about it.
These report messages are an excellent new feature to Duo but I think it's important to give it a little time see how they play out, how the users react to them and if it is successful and then think about ways to improve them even more.