https://www.duolingo.com/PatrikH

Duoling does not work. And a easy way to fix it.

For the last weak I have done nothing but practice weakest words, making sure to every day practice all the words with two bars or less. Of these words about 50 occur daily (I've made a list), and no matter how many times I get them right the decay rate does not improve. I have checked this by making a note of each word when it occurs.

This means that when I am through with all of these everyday occuring words (that I fact know very well), and would get into words that I in fact do not know that well I have usually already gathered about 100 points. Which in practice means I only practice the same words over and over.

Clearly something is wrong here, but without knowledge of the algorithm I cannot say what. I do know a way to fix it though, could we please get the option to "reset" a word, effectively making it "new" again, so that, when knowing it it would not decay so fast. It could be a button in the word page under "remove from practice"

This is serious, it has practically made duolingo unusable, because I cannot progress - either I go to the tree without actually learning the words (beacuse they will not repeat) or I just go through the motions of yet again translating the same phrases back and forth.

September 5, 2013

26 Comments


https://www.duolingo.com/tatou

I think I know what's going on here... and it's worse for people who practice a lot. :(

The word strength model combines a lot of info, including how "hard" the word is for people generally. So if you practice a lot, your "last seen" times and your past accuracies for words can sort of cancel each other out, and it "breaks ties" by preferring words that Duolingo users have trouble with on average (under the assumption that you will, too)... if that makes sense. I think that's what's happening.

I have several ideas for really fixing this (the "reset" button is a great idea, but also kind of a band-aid), but haven't quite had the dedicated time, since we're a small team juggling several projects each. And as Salxandra pointed out, changes to word strength can have drastic effects... so we try to tune things very carefully. :)

That said, Duolingo has a Hackathon coming up (http://www.duolingo.com/comment/774419), and I hope to spend that time improving word strength in some way. Stay tuned!

September 6, 2013

https://www.duolingo.com/cloclo15

This does make sense as i spend a much greater proportion of my time practicing (using the strengthen skills button on the app) than i do learning new lessons.

September 6, 2013

https://www.duolingo.com/PatrikH

Thanks, that makes sense - the reset button would be exactly band-aid, but I figure it could be made in a day, whereas tweaking the algorithm is probably more complicated. So, the way to avoid this for the time being is to not use practice latest words and try to avoid those words that decay too fast for some time.

September 6, 2013

https://www.duolingo.com/cloclo15

I couldn't even tell you the amount of times i have to translate al gatto piace il cane. Although i struggled with this when I first started learning, I have now translated this sentence correctly every time for about the last three weeks. And yet it still appears. Whereas i know there are other words that haven't appeared since i first encountered them. It is definitely odd.

September 6, 2013

https://www.duolingo.com/clearlywhite

I can 100% back up what PatrikH is saying. Until I get to around 100 coins, I only see words with which I'm relatively strong. However, after that point, it's like a switch is flipped, and suddenly I'm given all of the words that I am ACTUALLY weak with.

September 6, 2013

https://www.duolingo.com/PatrikH

This is exactly what happens to me too. I would not mind 100 points of "boring work" but when it does not amount to anything it is extremely annoying.

September 6, 2013

https://www.duolingo.com/richardliantonio

I have noticed this as well. another option would be to add a "show this word less frequently" rather than entirely remove it.

September 6, 2013

https://www.duolingo.com/cconvey

It sounds to me like the learning algorithm needs a tweak. One problem with manually over-riding the learnign system is that there may come a day when your overrides are inappropriate (suboptimal), but remain in effect anyway.

September 6, 2013

https://www.duolingo.com/Alia_La_Canta

That must be really frustrating! Note that what follows is a guesstimate of what's going on, since I have not seen the Duolingo algorithm, and in fact rarely use "Practice weakest words" for different reasons to that described by PatrikH:

I think if you always practice words in the same order, and they have the same decay rate, they always appear in the same order when next you visit. This is especially a problem for words that are rarely wrong. The trouble with language - or at least actively-learnt languages - is that most of the time, a learner will get most words right most of the time. They will only be struggling with a fraction of the language at any given time. This means the usually-right words, which algoritms find most difficult to sort due to a relative lack of data, predominate.

This gets worse because a word struggled with at one time may not be struggled with forever (and in fact usually won't). Statistically, it's difficult to iron out the effect of originally not knowing the word, and it will always flag up as "weaker" relative to the other words learned with less effort. So it may be that the test isn't testing the weakest words at a given time (which is what most users probably want when looking to practice "weakest words"), but testing the weakest words on average through a user's time on Duolingo, with possible slight bias towards words that were difficult 3-12 weeks ago.

It's a very difficult one to sort out - there's probably a graph that could be drawn of a typical user's retention patterns and half-lives of errors, but computers are not good at processing graphs. They like straightforward numbers and linear equations. I am hopeful that Duolingo will be able to iteratively improve the procedure.

I don't use "Practice weakest words" because at the moment I don't know enough words for the "Time to practice" ones to be difficult to manage (even in Italian, the language I've studied the most on Duolingo, I rarely have to practice more than 10 in a week). So I can simply sort the words by "Strength" and do all the ones that are considered weak for all forms of the word. If I make an error on a word more than once in a day's lessons, I do separate practice on that word regardless of indicated strength.

Fixing the algorithm for "Practice Weakest Words" would be ideal, but if the situation is as I suspect, that's likely to be a long-term project. In the interim, there may be something you can do to help yourself. Write a list of all the words you get wrong each day in lessons (and, where possible, those you have trouble with in Immersion), and practice them separately in Vocabulary. There's a search box in Vocabulary where you can type each word and save yourself the bother of loading every word you know.

It may also be worth randomly selecting a lesson category or two each day, and practising each word you can't instantly define and use. This changes the order you cover words, and may help the algorithm put some unpredictability into your practising. Even "Practise Words You Haven't Seen For A While" beats "Practise Words In Sentences You've Already Memorised".

September 6, 2013

https://www.duolingo.com/PatrikH

Ok, I'll start with your solution, in short, tried it does not work - I practised one word about 20 times, no change.

Initially my estimate of the situation was the same as yours, but now I'm thinking it is only part of the problem. It seems that there is really a bug involved, because if the case was as you describe there would be at least slight changes - and I have mad notes of all these words for a week now and most of them don't change at all, even though I get them right each time.

And it does not seem that it is difficult to fix this in the algorithm - if the case is that it calculates decay rate based an all occurrences of the word. The solution is simply to take into account only the latest 20 or so occurrences and ignore older one's. But this is all guesswork, it would be really nice with a comment from the developers.

September 6, 2013

https://www.duolingo.com/Alia_La_Canta

So, you practised a word 20 times* and it still ended up in the same point of the order? That's bad.

The "making notes on the ones you get wrong" should still be useful from a learning standpoint: it means you practise the words you need rather than the words a computer thinks you need. If this is not making any difference to the practise sequence, then things are worse than I thought from the initial description. "Most of them don't change at all" would imply slight changes are happening, though whether this is a sign of my hypothesis being correct or sheer coincidence is difficult to tell. I'm inclined towards the latter, given the other things you have said.

    • I'm assuming that this was a word where 20 practices would consistute a decent proportion of the times the word was seen.
September 7, 2013

https://www.duolingo.com/plurallingo

If you click on a specific word in your vocabulary you can click "remove from practice" right under the buttom that says "practice word".

September 5, 2013

https://www.duolingo.com/Medvidek

But is there a way to put it back afterwards? Because I am afraid that if I hit the remove-button, I will "lose" the word forever...

September 6, 2013

https://www.duolingo.com/lmarcetic

I'd like an answer to this. Nothing visually changes when I hit the button. I don't know whether a word is hidden, and I cannot bring it back if it was removed (since "remove from practice" button remains).

September 19, 2013

https://www.duolingo.com/tatou

There is not a way "un-remove" words from practice right now, but this is on my short list of features to add, so look for it soon.

September 19, 2013

https://www.duolingo.com/IG88
  • 2338

He knows that and says so in his post. I think he's suggesting a better solution than just removing oft-recurring words from practice altogether.

September 5, 2013

https://www.duolingo.com/Salxandra

Eek eek eek. You know, every time Duolingo "fixes" practice weakest words, my whole tree loses its golden leaves. Sigh. I guess its time because I've almost gotten my tree gold again.

September 6, 2013

https://www.duolingo.com/adaduv4

I'm about ready to give up. I'm stuck on the first Spanish Lesson, Basics 1. I've practiced these nine words over and over again, and now I'm on level 4, but to get to level 5, I have to practice these nine words a hundred times!? And there's no other way to get out of this section that I can see - you can test out of the entire lesson but I'm not ready for that. Very frustrating! (I hope this is related to what you are discussing - my apologies if I'm off-topic).

September 6, 2013

https://www.duolingo.com/Salxandra

Oh, I'm so sorry. It sounds very frustrating.

Usually, what has happened is that you're clicking on the same lesson over and over. When you open up Basics 1, you should see at least 2 blue notebooks. You have completed the blue notebooks with check marks. You likely have a blue notebook that has a gold coin with the number 10 within. You have NOT done that notebook.

You need to get all of the notebooks for a lesson completed (check marked) and then, Basics 2 and Phrases will become available to complete.

By the way, welcome to Duolingo.

If you have any other questions or if the above didn't help, please provide additional information so we can help.

September 6, 2013

https://www.duolingo.com/adaduv4

Ah, that was it - I've been using the link that Duolingo sent me via email which takes me straight to the practice page for the first notebook. I didn't even know there were other notebooks in the basics 1 section. Thanks for your help.

September 6, 2013

https://www.duolingo.com/Alia_La_Canta

If you've accumalated enough points for Level 4 while waiting for the second lesson, you must have great patience and tenacity. Keep up the good work, adaduv4 - I have a feeling you'll be a very good linguist by the time you finish the tree.

September 7, 2013

https://www.duolingo.com/Salxandra

Hopefully, one of the Duolingo experts will notice your comments because it seems to be a frequent issue right now. Maybe they could update the program so it's more obvious what needs to be done.

By the way, I just deleted my German progress and redid its Basics 1 to make sure I wasn't missing something. That was fun. It confirmed for me that you really shouldn't have only half the lesson done and be level 4 already. You've really been repeating those lessons. Believe me, the rest of Duolingo is going to be easy peasy for you.

P.S. Congrats, I've just noticed that you've got that third gold bar for Basics 1. Just one more gold bar and that section is done, and Basics 2 opens. :)

September 6, 2013

https://www.duolingo.com/DeeRamm

I thought that doing translations was supposed to improve your word strength (although I'm not sure where that is right now). When I finish a Spanish lesson, it usually gives me the option of working on a translation. If I go to the translation via that button, it makes a point of keeping track of words I've strengthened. I'm assuming that stops the "decay" of my Spanish vocabulary. (I'm not sure that happens when you translate via the "immersion" button.)

Strangely, for German, that option was only presented for a few lessons and seems to have gone away again. (I opened a support ticket on that, but have never heard what is going on.)

Anyway, I think that, if working right, this would solve your problems. Instead of repeating the same stuff again and again, you could be looking at and doing translations. This can be a lot more fun than rote repetition and you get a lot of control over what you translate, or, in many cases, approve or improve someone else's translations.

September 7, 2013

https://www.duolingo.com/clearlywhite

I think they fixed this... Is it working way better now for anyone else?

September 9, 2013

https://www.duolingo.com/erikpukinskis

I have the same problem. The "practice weakest words" almost seems like it gives me the strongest words. Words I get wrong over and over rarely ever even show up.

I've taken to practicing the specific skills that I know I'm weakest at, as a band-aid, but it'd be great if Duolingo could fix this.

Part of me suspects their algorithm is just too smart for it's own good. I suspect a dumber algorithm that just gave you words to practice based on how long it's been since you got it right would be way better. E.g.:

  • put a word in the practice pool
  • get the word wrong - keep it in the pool
  • get it right - take it out of the pool for a day
  • one day elapses - put it back in
  • get it right a second time - take it out for 2 days
  • 2 days elapse - put it back in
  • get it right a third time - take it out for 4 days
  • get it wrong in any of these situations - go back to the start

and so-on. But that's just me.

September 9, 2013

https://www.duolingo.com/PatrikH

yes, I kind of agree - for example having the algorithm take into account other peoples progress must give bias to words early in the tree for example, since so many people quit after a while. And people with different first languages will have difficulty with different words and so on.

September 9, 2013
Learn a language in just 5 minutes a day. For free.