Duolingo is the most popular way to learn languages in the world. Best of all, it's 100% free!

https://www.duolingo.com/olimo

New words decay: continued

olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

This is a little follow-up for this discussion: http://www.duolingo.com/comment/979823 where I wrote that newly learned words should decay in a day or less so that they would be strengthened in time and not forgotten altogether by the time the skill decays.

Disclaimer: This post is mostly intended for the team. I appreciate opinions and experience of other users, but please don't suggest workarounds or explanations why this can't be fixed right away - I'm well aware the team is small and the tasks are huge.

Two days ago, on the 4th of November, in the evening, I learned the "Places" skill in Spanish. This was not a very difficult unit for me because many words look like those in English or French. I decided to do an experiment and not to practice this skill right away or even next day and see when it decays.

Almost two days passed. What do I see in my Vocabulary for "Places"?

Places

Only nine words out of 63 decayed. Nine words. This is not enough even to decay one lesson out of 4, and of course the whole skill stays gold.

I don't feel I really know those words so well. Of course I will be able to translate sentences with them from Spanish into English, peeking occasionally. In fact, I've already done it once - when I passed the lessons for the first time. I usually manage to pass a lesson from the first attempt, but that does not mean I know the words perfectly! Being able to translate by peeking is not knowing the words. For me, remembering a word really well is being able to translate it from English to the target language and doing it without peeking and without hesitation. I doubt I can do it for every word from "Places".

Let's do a little math. There are 9 lessons in "Places", 20 sentences in each of them. This makes 180 sentences. And 63 new words. This means I've seen every word 3 times on average. 3 times! This is not enough to learn the words. I still have to learn them, but the system thinks I know them and only have to practice these words once in a while. In a week, maybe?

Another issue I want to raise is the difference between new words from the same skill. I believe "playa" got decayed most because I made a mistake in one or more sentences with it. I don't remember exactly, but it was most likely a mistake of the phrasing, i.e. I could have typed "at the beach" instead of "on the beach" or something like that. This hardly means I have more problems with "playa" than with other words. In fact, when I look at the words, I don't remember what "hogar" means and I doubt I would type the Spanish word for "West" (oeste) correctly if asked to do it. My point is that new words should decay with approximately the same speed because the main reason for their decay should be the very fact of them being new and not the mistakes made doing lessons.

How do I see a perfect decay situation for this skill? I should have woken up the next day (yesterday morning) and have found the whole skill at 3 bars. It should have been 2 bars by yesterday evening if I had no time to practice in the morning, and 1 bar by now. The whole skill, not only "playa", needs a refresh. And this refresh should be not one practice session but enough sessions to cover every word and ensure it is strengthened.

I have a theory why the system does not work well for me. Maybe that's because I usually manage to pass lessons from the first attempt, very often with all 3 hearts left, and the system thinks I already knew the words and grammar taught in this lesson? But that's not true. Although I listen to Michel Thomas's courses and am somewhat familiar with the grammar, I still have to learn the vocabulary, and most of the words are really new to me.

tatou said:

I agree that if new words decay faster, this is theoretically better for learning. However, we've experimented with this a little but and found that most users find the quick decay rates de-motivating because they are more likely to quit.

Well, maybe I am strange, but I'm more likely to quit if I progress happily through the tree and then bang! some of my old skills decay and I suddenly realize I remember almost nothing from those skills. In my opinion, it is more frustrating to feel stupid than to do a little more practice earlier.

Maybe there should be some way to make the first practices look more like a part of lessons? Before, there was "learning" and "mastering", and people had to practice or translate to master their skills. This was not seen as decay and returning back, but as a part of learning. What about incorporating some mandatory or highly recommended practice right into the lessons? Seriously, getting a gold icon and a "learned" status for only seeing every new word three times is too much a stretch.

4 years ago

36 Comments


https://www.duolingo.com/tatou
tatouPlus
  • 19
  • 11
  • 8
  • 2

Thanks for the detailed feedback, everyone!

First: we have a new word strength model we are planning to roll out this week to everyone (first mentioned here: http://www.duolingo.com/comment/997120). We've experimented with it for 2 weeks and it fixes several of these issues for most Duo users, and users tend to come back and use the site more with it, as well as complain about word decay less in the forums (particularly active folks at more advanced levels). So it seems to be working! :)

However, olimo is one of the users in our experiment who already have the new model, so that doesn't explain why she's seeing wacky behavior. Mathematically, it's impossible with the new model to have 4 bars after 2 days if you've only seen a word/concept once. I looked into it, and here are a couple of things going on:

  1. According to the model, olimo has actually seen all the words in "places" at least 4 times, and gotten them correct nearly all of those times, and most were seen 8 or more times ("playa" was correct 2/8 times, which is why it's so much lower). Most users do not do an entire skill in one sitting, so multiple exposures to the same word in the same skill get spread out over time, and the model statistics are fit to data from all users accordingly. In olimo's case these words were all seen essentially at the same time, which is rare and not accounted for in the way we "train" the model. So we'll rethink how we keep track of these tallies to account for more "bullet-train" users like olimo.

  2. There were some hacks I wrote over the past few months (http://www.duolingo.com/comment/678585, http://www.duolingo.com/comment/878880) which sometimes override the word strength model if you see a word many times and are pretty much always correct (to detect words you probably knew before you started Duolingo, or cognates that you don't need to practice much), and slows down the decay. I suspect that it's unnecessary with the new model we're rolling out, but it is still there, and is triggered for about half of olimo's words in the "Places" skill. Disabling it might (hopefully) improve things.

It's quite possible that everyone else chiming in this thread are affected by the same problems. So, these are both bugs we'll deal with ASAP, and I think will fix your problems, olimo. However, these changes require a lot of care since they can have massive effects for millions of people.

There are several other improvements I'm working on as well, which should address olimo's concerns about "playa" vs. the rest of the skill — that is, the way in which you make an error, not merely the fact that you made one. Unfortunately, this is not a simple hack. I have in fact started on this several times over the past few months but haven't finished due to more urgent things popping up. So rest assured that we're aware of these problems and actively working on them.

I'm sorry I don't have time to reply to every point made in this thread right now (I just got back to the office from a trip). In the meantime, I'm going to get busy fixin' stuff! :)

4 years ago

https://www.duolingo.com/_pinkodoug_
_pinkodoug_
  • 25
  • 11
  • 11
  • 7
  • 6

For me, things have actually improved since our discussion that you linked in your second example. That is not to say that there's not an issue with new words decaying too slowly, just that at the other end of the spectrum the problems with well known words decaying too quickly has improved quite a bit, though not entirely.

For example, yesterday one of the words I practiced was "el fenómeno" (phenomenon). This is a word that for some reason continues to decay much more rapidly than other words. After practicing, it was 3rd from the top of my vocabulary when sorted by strength, indicating that its strength at that time was higher than 1568 other words. Today, not even 24 hours later, it's 50th from the bottom and weaker than 1521 other words (though still 4 bars, for now). Something is definitely hinky with the decay rate for that word.

As an aside, I would have liked to have tested your word strength fix, but unfortunately I didn't bother to ask since my time has been pretty scarce these past 2 or 3 weeks due to a couple of projects here, and I didn't feel that I'd be of much use in testing. I'm just about through the worst of it with those, so hopefully I'll be back to my normal activity level in the next few days.

Edit to add: Less than an hour after making this post, "fenómeno" has decayed from 50th from the bottom to 42nd from the bottom, putting it on track to be my weakest word by later this evening, a swing of 1568 places in less than 24 hours despite my correctly encountering this word many, many times.

Update: 2 hours later it's 8th from the bottom, putting in on track to be my weakest word by later this afternoon.

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

This is a very detailed outline of the problems, I think it covers the problem rather well. I know it probably takes time to implement solutions to resolve these issues. One can only hope the new model tackles the problem effectively.

In the meanwhile, is there any way we could maybe have something like "Practice last XXX(10) words learnt button"? I think this would be a quick hack, and would would allow us to refresh the last few words learnt, without rummaging through the vocabulary section, and clicking each word to practice individually. This would be useful because in some cases some words (e.g. in German) are so complicated because of the spelling that I forget them by the time the lesson is over. However, the algorithm thinks I know it since I saw it recently.

4 years ago

https://www.duolingo.com/ariadna.m

I agree with the "Practice last XXX(10) words learnt" button. I think It would be a very good idea and it would actually stimulate people to practice words learned recently.

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

Thank you for such a thoughtful answer, tatou. I understand it's not easy to guess every user's actual level from his/her answers in lessons, that is why I'm eager to give information that shows how some of the algorithms might not work.

  • While it is logical to suppose that the user is likely to forget words in which he/she made mistakes, this is not always so. I remember "playa" perfectly, but I don't remember "hogar" or "esquina", for example. Here's a suggestion: What about not counting mistakes and times of words seen during initial lessons but rather test all these words after a certain time? Let the user only translate into target language and see what they can remember without peeking. Then, schedule the next practice according to the results of that test.

  • Cognates are easy to understand but they still need some practice to remember the exact spelling and gender. Of course, "avenida" looks like "avenue" and is easy to understand, but if asked to translate "avenue" into Spanish, you can still stumble on it. Something like "avenue", yes, but what exactly was it?

  • When a word is first introduced with a picture, it is easy to get it right without peeking, and then the short-term memory steps in and often allows me to get this word correct during the whole lesson. The system may conclude that I've known this word previously - and be wrong. That's what I believe has happened to my "hogar", "esquina" or "carretera". I really saw them for the first time in lessons and don't recall the meanings as I look through my vocabulary right now.

You know, sometimes it seems to me that it is so hard to program spaced repetition of words and take into account mistakes of grammar, syntax, inattentiveness and so on, that I think it could be really better to use somewhat "dumber" methods like introducing some mandatory practice and tests after it. Or, at least to make practice available any time in the app as it is in the web version so that users could use their devices to practice as they like if the system does not work too well.

Anyway, thanks for your hard work and please don't think I'm just whining :-) I'll most likely continue to post my thoughts on this topic hoping they would help to determine what assumptions are right and what don't seem to work so well. I'm looking forward to see the changes and willing to participate in any tests :-)

4 years ago

https://www.duolingo.com/OldSpiceGuy
OldSpiceGuy
  • 25
  • 25
  • 13
  • 5
  • 4

Excellent post. It covered everything I was going to say. Cognates are easy to translate out of a language. If I can get a word after a period of time translating INTO the language, it should be gold.

4 years ago

https://www.duolingo.com/pont
pont
  • 25
  • 23
  • 18
  • 16
  • 14
  • 14
  • 11
  • 9
  • 7
  • 5
  • 373

Thanks for the update! Your hard work is appreciated, and I appreciate the need to be cautious when rolling out changes to such a large user base...

4 years ago

https://www.duolingo.com/Learnerplates

Well I am completely overwhelmed by the amount of practice I apparently now need to do - I have worked exrtemely hard this week and was thus horrified this morning to discover the strength of hundreds of words down to 2 or 1 bar. And I do mean hundreds, that's not an exaggeration.

Is it likely that the new tweaks make it more difficult the further up the tree you are, or the more skill points you have? I don't see myself ever being able to get through this.. more than 350 skill points today and it's not getting me anywhere. I feel like I have just been penalised for working really hard every day for the last 3 months.

I'm rather despondent, to be honest :-(

4 years ago

https://www.duolingo.com/Ursulias
Ursulias
  • 25
  • 25
  • 25
  • 23
  • 23
  • 21
  • 20
  • 11
  • 6
  • 751

Sorry to use this way to contact a DL Admin. I just came across your name and could see how often and how well you help users. I have no idea of the DL structures - but maybe you can help. I guess all of the Admins see the upsetting discussions about the last big update and the destruction of what had been the great fun in practicing with DL. The crowns update makes DL boring for advanced learners. I cannot imagine how something like this could happen. Can any Admin explain what is going on? Is DL trying to get rid of advanced learners? Is it a try to increase the minimum time people have to spend for DL - in order to increase the income via advertisement? - I would really appreciate some official statement from DL to the devastating update. It has been such a great app - and it makes me sad to see what has happened.

5 months ago

https://www.duolingo.com/Bigface

I'm sorry that you're "devastated" by the crown update. But this has been a huge boon for me. You seem to think that change is a bad thing, that everyone agrees with your assessment. But I just want to reply and confirm that is not the case. Maybe they could have options that let you include decay or something, so that you advanced learners can still practice they way that you're comfortable. I get that you don't like the changes, and looking at your impressive levels and streak numbers, you should have more weight with your opinion than I. But, sometimes changes happen that the majority like (I'm sure DL's internal numbers will confirm or deny this with time), and you don't. And when that happens, you have to either adapt or leave the program.

4 months ago

https://www.duolingo.com/FrankySka
FrankySka
Mod
  • 25
  • 23
  • 21
  • 16
  • 14
  • 10
  • 8
  • 1884

There appears to be a divide between users wanting things to decay slower vs faster. I am not sure whether this is due to different decay rates being tested, or whether this is a real thing, but this is the impression I get from various discussions. I am not normally an advocate of having an option to choose one vs the other (probably a lot of work to 'maintain' for the duo team), but for this issue I really think it would help.

While I agree that subjective assessments of what the best way to learn might not always be valid, I think having so little control over things is really frustrating. I have said it before and I say it again. This is precisely why I am so annoyed that I cannot practice golden skills on the app anymore (without repeating the single lessons). Whenever I do go back, I hardly pass the lessons that I apparently have full strength at. That just cannot be right.

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

From what I get, skills should decay faster at first and slower after a significant period of time. People usually complain about being unable to properly learn new words or about being suggested too often to revise very basic skills learned long ago.

4 years ago

https://www.duolingo.com/BohemianCoast
BohemianCoast
  • 19
  • 16
  • 12
  • 12
  • 10
  • 6
  • 6
  • 2

The thing that particularly annoys me about decay rates is that, of everything Duolingo is trying here, it's the one thing where we have solid, thorough scientific research. SRS works for vocabulary retention, and the algorithms to implement it are well-understood and in the public domain. People argue around the edges about questions like the exact approach to decay rates based on individuals' success rates or item success rates, and whether you need a different approach for a few 'hard to learn' words, but the basic methodology is sound. And Duolingo's decay system is nothing like it at all.

Though having said that, nothing in Duolingo is as irritating as the failure to tell you what you should have done when your time runs out in timed practice. And I guess that would be a quicker fix than sorting out the entire decay system.

4 years ago

https://www.duolingo.com/pont
pont
  • 25
  • 23
  • 18
  • 16
  • 14
  • 14
  • 11
  • 9
  • 7
  • 5
  • 373

I agree strongly with your assessment. For the past couple of weeks, I've been increasing the pace of my Italian learning (1-3 new skills per day) and all that progress is still solid gold. For example, looking at "Verbs: Imperative", I see a whole list of imperative forms which I learned two weeks ago and have not practised since then. They're still on four bars! This does not seem like sensible spaced repetition: after two or three weeks it's too late to refresh anything I saw for the first time in that skill. It's forgotten, and I have to relearn it.

Like you, I am mitigating this by using other resources: Michel Thomas, Anki, reading and conversation in Italian, etc. -- so actually I partially know much of the grammar and vocabulary before I see it on Duolingo. (Also, the strong predominance of Italian-to-English vs. English-to-Italian means I can guess most of the vocabulary I don't know.) But if I were relying to Duolingo alone I think it would be very frustrating.

In contrast, Anki's default SRS configuration gives me a first repetition within 10 minutes, another the following day, and then I think maybe 3 days... but those first two repetitions are crucial to my learning.

From Tatou's comment, though, it seems that this is a conscious decision. I do appreciate that Duolingo's priorities here are not quite the same as mine (or yours, or Anki's). I am already motivated to learn the language and want to do it in the most effective way possible. But Duolingo is trying to motivate casual language learners as well as helping committed language learners: they have to deal not only with "spaced repetition vs. massed repetition", but also with "any kind of repetition vs. another game of Angry Birds". Better, perhaps, to have 10 million users learning slowly than 5 million users learning quickly. If this is the reasoning, I won't argue with it -- Duolingo have the user data to make these judgements, of course -- but it's a shame that the system has to be made less effective in order to make it more attractive.

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

Motivating casual learners by lower decay rates is a mystery to me. I try to imagine myself a careless casual learner who is not very motivated to use other resources or plan his/her own schedule. What I see is that I do a lot of new lessons because it's more fun than practice and get stuck somewhere at the point of 1/4 or 1/3 of the tree because I forget a lot and new skills get bigger and harder. I wonder what the goal is: to retain users for 1/3 of the tree or to make sure more users complete the whole tree...

Again, for this, I suggest making more practice part of the regular program so that a casual user would not be demotivated by something decaying too fast. Losing something (even as petty as the gold color of icons) is always unpleasant.

4 years ago

https://www.duolingo.com/pont
pont
  • 25
  • 23
  • 18
  • 16
  • 14
  • 14
  • 11
  • 9
  • 7
  • 5
  • 373

Indeed, it's a mystery to me too. As you say, making it easier initially would surely just make it harder later. A lot of the functionality has changed during Duolingo's short life, so we can only hope that this too will be reconsidered in time.

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

There is no mystery, casual users are short-term oriented(more rewards now), committed users are long-term oriented (more rewards later). To put it bluntly, a casual user will not worry that much about the consequences of not practicing, committed users will want to ensure they grasp the basics effectively so they will reap the rewards later.

4 years ago

https://www.duolingo.com/mbagnall
mbagnall
  • 25
  • 22
  • 6
  • 5
  • 4
  • 4
  • 1607

It's interesting that what you are experiencing is quite the opposite of what I do. I'm finished my Spanish tree but I do constant practice to reinforce what I have learned.

Unfortunately, words in my tree decay extremely fast making it almost impossible to keep things 'gold'. Words that I have seen lots of times like 'para' (419 times) or 'no' (1890 times) are currently at 1 bar out of 4. This is despite having last seen them 16 hours ago!

It makes it very frustrating for learning/revising since it's difficult to identify and focus on what are really my weak areas vs. what is being reported as my weak areas.

I hope the team can fix these issues but since they differ for various users it'll probably take some time :-)

4 years ago

https://www.duolingo.com/pont
pont
  • 25
  • 23
  • 18
  • 16
  • 14
  • 14
  • 11
  • 9
  • 7
  • 5
  • 373

I experienced something similar in my German tree. If I remember correctly, the explanation (as given by Tatou) was that I had at some point made a lot of mistakes with certain word forms, and a word strength bug had then somehow locked them in at 1 bar, no matter how many times I get them right afterwards. Last I heard, there was a test under way for an updated algorithm which fixes this bug. For now, the best workaround is probably to manually remove these words from practice.

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

It seems that the grass is not greener on the other side.

My guess is that you are part of the experimental group in which words decay quickly. Besides, if Spanish is anything like portuguese those two are high frequency words, and are included in many sentences. Tatou mentioned that sometimes we are actually practicing a different word, and the sentence just includes a word so it can make sense.

Either that or you are experiencing that 2 month decay problem ( which pinkdoug described), and words suddenly start decaying after some time. My advice, is that you either take the certificate test to see how well you score, or reset the language, and take a placement test.

4 years ago

https://www.duolingo.com/itastudent
itastudent
  • 17
  • 14
  • 10
  • 9
  • 2

I agree that the word decay algorithm could be improved, but personally I am among those people who wouldn't like to see half tree decayed in a couple of days without practice.

Also, I think Duolingo could be never able to know exactly which words I know better... he can just try to guess based on the information he has. By the way, I already practice language out of Duolingo, sometimes I read article in the languages I'm learning, I talk to native speakers, etc. A lot of times, I could have seen a word just once, and I could remember it for a lot of time because it's similar to the same word in other languages I already know.

In my opinion, the best way to practice would be to let the users choose which words they don't know (who better than you can know if you know a word?). You could think this as a separate mechanism from the current practice Duolingo offers, or they could be integrated together.

So, I would like to pick up the words I would like to practice from the vocabulary sections... I would also like to have the possibility to add words to my personal list of weak words while I'm doing an exercise. For instance, I get a sentence where there's a word I don't really remember and I think "What the heck does that mean?!?"... I would like something like Right Click on the word -> Add This Word to My Weak Words. Next time I will practice my weak words, Duolingo will know that is a word I like to practice, because it got my suggestion. Based on users suggestions, Duolingo should be able to pick for you the words you really like to practice.

4 years ago

https://www.duolingo.com/oskalingo
oskalingo
  • 25
  • 15
  • 13
  • 11
  • 10
  • 7
  • 4
  • 3
  • 2

Olimo, by way of comment I'm going to quote from a study into the effectiveness of massing (repeated revision in a short span of time) vs spacing (the spaced repetition system that duolingo uses) and, relevant to your post here, the validity of a learner's own intuition on what is the more effective method for them. Please don't take this as me patronising you, you're obviously a much more experienced language learner than me. But my understanding is that duolingo follows an empirical approach in trying to optimise our learning by doing a lot of number-crunching on our progress and performing AB tests on different approaches, rather than so much relying on our feedback on how we think we learn best (which might actually be wrong). Of course, this asks for a lot of trust from us. Anyway, here's the quote:

Contrary to the massing-aids-induction hypothesis, final test performance was consistently and considerably superior in the spaced condition. A large majority of participants, however, judged massing to be more effective than spacing, despite making the judgment after taking the test.

…Metacognitive judgments - that is, judgments about one’s own memory and cognition - are often based on feelings of fluency (e.g., see Benjamin, Bjork, Schwartz, 1998; Rhodes Castel, 2008). Because massing naturally leads to feelings of fluency and increases short-term task performance during learning, learners frequently rate spacing as less effective than massing, even when their performance shows the opposite pattern (Baddeley Longman 1978; Kornell Bjork, 2008; Simon Bjork, 2001; Zechmeister Shaughnessy, 1980). Averaged across Kornell and Bjork’s (2008) experiments, for example, more than 80% of participants rated massing as equally or more effective than spacing, whereas only 15% of participants actually performed better in the massed condition than in the spaced condition.

Kornell, N., Castel, A. D., Eich, T. S., Bjork, R. A. (2010). "Spacing as the friend of both memory and induction in younger and older adults". Psychology and Aging, 25, 498-503

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

I am not advocating for massed repetition! I just want the spaced repetition to be properly adjusted.

the spacing effect is the phenomenon whereby animals (including humans) more easily remember or learn items when they are studied a few times spaced over a long time span ("spaced presentation") rather than repeatedly studied in a short span of time ("massed presentation")

(from Wikipedia, just in case - I had to check what "massing" exactly is)

I'm not saying I should review the skill the next day and then drop it. I'm saying that the first practice should come earlier and a few subsequent practices should also come with smaller intervals. You see, while spaced repetition is proved more effective, the exact intervals of practice are not stated anywhere. Here are two schedules (just examples):

  • learn → wait 4 hours → refresh → wait 12 hours → refresh → wait 1 day → refresh → wait 2 days → refresh → wait 4 days → refresh → wait 7 days → refresh → ...

  • learn → wait 3 days → refresh → wait 5 days → refresh → wait 7 days → refresh → wait 10 days → ...

Both are spaced repetition, both increase intervals but never end, but the intervals are different. With the first schedule, no one can make the student to practice 4 hours after the initial lesson, but it is encouraged. If you don't have time to practice, you'll do it later, that's all. The next interval will still be 12 hours - you'll be encouraged to practice after 12 hours. If you feel confident or don't have time to practice, you can postpone it, but at least you will be aware that you are likely to need it.

With the second schedule, the user is not encouraged to practice until 3 days pass. The user may rest assured he/she remembers the words while in fact it is very easy to forget them in 3 days, especially if there were a lot of new words. After 3 days, the skill decays and the user suddenly realizes he/she remembers almost nothing, feels stupid and frustrated. He/she does practice, recalls the words and then again is made confident he/she knows them. After 5 days, again, he/she struggles with these words again and it is almost as hard as it was for the very first time.

If a user chooses to practice earlier than it is recommended, the intervals should not increase faster. I.e., if I make a long practice of my lessons, nothing from scheduled practice should be skipped, and I still should be encouraged to practice in 12 hours or in a day or as it was planned.

Another thing that I want to emphasize is that the initial lessons are too short and contain too many words. If you spend an hour learning and practicing just 10 words, you'll be able to recall them after a longer interval, and the first interval for spaced repetition can indeed be longer. Right now, nothing prevents users from doing, say, 10 lessons in one sit and end up with 70 new words that make a mess in their head - and a few days until first practice is recommended. If the lessons were longer, people will only do one or two lessons and take a rest, thus ending up with less words and less mess.

The current situation results in many users (including me) doing practice using their own schedules, and of course it is tough to manually set intervals for skills and use your own spaced repetition. That is why discussions like this one appear again and again.

I know I'll learn everything I'm supposed to anyway, but I just want Duolingo to help me better with my learning schedule.

4 years ago

https://www.duolingo.com/BohemianCoast
BohemianCoast
  • 19
  • 16
  • 12
  • 12
  • 10
  • 6
  • 6
  • 2

Duolingo isn't using SRS in any recognisable way; that's actually the problem. If you make a mistake, you see the same word four times in the next five minutes, and then it puts you back to four bars and doesn't show it to you again for a month. That is absolutely not how SRS should work.

4 years ago

https://www.duolingo.com/FrankySka
FrankySka
Mod
  • 25
  • 23
  • 21
  • 16
  • 14
  • 10
  • 8
  • 1884

I love that we cite research articles now (I am not being sarcastic!). An important point is however that spaced repetition ONLY works when some initial learning has taken place. olimos point is exactly that, that after finishing a lesson not enough initial learning has taken place. You can have the best algorithm in the world, if there is nothing that can be forgotten in the first place, it does not work. :)

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

Personally, I'd actually prefer if more people cited research articles, at least in discussion where it is warranted like this one. But anyway, I'm a research student, and others may not like it.

4 years ago

https://www.duolingo.com/FrankySka
FrankySka
Mod
  • 25
  • 23
  • 21
  • 16
  • 14
  • 10
  • 8
  • 1884

well if at some point we ever have a more organized discussion (with different subsections) we could have our own little nerdy sections of articles :) (Disclaimer: While I think the discussion should be organized at some point, I do not care much about it currently)

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

This is a rather verbose text, and I understand the need for it. But in my opinion, it is often better to summarize or put subheadings to make it easier for people (Staff or users) to understand your ideas. From my understanding you seem to have 3 main issues, skill decay, practice, and outside knowledge.

Regarding the skill decay problem, tatou has acknowledged it, they are trying to balance motivation vs effective learning. The trouble with that is that we aren't all motivated by the same things, and I think the model should evaluate individual progress and decay it more/less depending on that. So you're a victim of your own success, to beat a lesson with 3 hearts you have to be either good in the language or peeking a lot(cheating).

The practice system is not using the best approach in my opinion, when practice is optional, users will ignore it until they really need it. Lessons are used as a means of practicing and testing, so if you fail enough of those you'll know your skills are lacking, and go back to real practice.

The external knowledge is in my opinion what makes you proceed so swiftly, and also what prevents the system working well for you. You have prior knowledge which makes you pass lessons with relative ease, the system will either assume you're fluent or have some knowledge and don't need much repetition.

As test I tried starting Spanish, and did the basic lesson 4 days ago, and I flew through it with very few mistakes. My whole vocabulary is like this (http://i.imgur.com/6ahwNKS.png). At this point, in my opinion it is an accurate measure of my skills since I know Portuguese, and Spanish is easy for me to understand even without knowing it well.

I agree that the word strength system needs to be improved, but I think in your specific case, what you need is a custom practice. This would ensure that you can practice just the words you don't understand, because your external knowledge makes it hard for the system to pinpoint what you have trouble remembering.

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

Thanks for the remark about the text. I'll try to highlight some key points to make it more readable.

So you're a victim of your own success, to beat a lesson with 3 hearts you have to be either good in the language or peeking a lot(cheating).

Peeking is not cheating, it is like using a dictionary; and a lesson is not a test where you are not supposed to use dictionaries or other references. Also, being successful at lessons does not mean being able to use new words in real life without difficulties. Guessing what a given Spanish word means is much easier than remembering the right word when you have to use it. Lessons usually give little tasks to translate from English into the target language, so they can't be a very good measure of your knowledge.

Also, I don't only care about myself (I can cope by ignoring the algorithm and using my own schedule instead), but I see a lot of reports about skills decaying too slowly and realize I'm not the only one who has this problem. With this post, I wanted to give an specific example to illustrate the shortcomings of the current system.

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

Well, that's true, the issue is that duolingo does not really offer any lessons. A lesson teaches a concept. It is #period of learning or teaching, and what we do here is exercises or tests (# the means by which the presence, quality, or genuineness of anything is determined; a means of trial) with some hints.

As I've suggested before what we truly need is real tests/exams such as the certificate test. The whole notion of completing a skill and never being tested on it again except on random practice makes no sense to me.

Another thing Duolingo lacks is a full test relying only on the target language. Questions and answers asked in the target language will provide a better measure of your knowledge rather than translating something back and forth. I mean if you complete the tree, you should be able to answer at least a grade 1-3 test.

I entirely agree with your previous point, practice should be integrated in a seamless way within the lessons. I think one way to achieve this, is some sort minimum score such as needing at least 60% accuracy on practice lessons before starting a new skill.

4 years ago

https://www.duolingo.com/olimo
olimo
  • 22
  • 22
  • 20
  • 19
  • 10
  • 9
  • 8
  • 8
  • 7

In fact, I like Duolingo's method of learning through exercises without much formal teaching. This is what makes Duolingo different and fun.

I absolutely agree with you that there should be some practice integrated with the lessons. For those who don't need this practice because of being really bright or having some previous knowledge, there should be a test where you have only to translate to the target language and without any hints.

at least 60% accuracy on practice lessons before starting a new skill

OMG, this is too easy! With that score I'd be a victim of my own success again :D

4 years ago

https://www.duolingo.com/Dessamator
Dessamator
  • 25
  • 14
  • 13
  • 13

Such tests already exist, placement tests/test-out of skill. Regarding the 60% accuracy, it all depends on how it is presented. For example, 60% of 100 words is quite challenging to anyone who hasn't grasped a language.

If the lesson practice tested all or at least 80% the previous skills words, regardless of word strength then it would be worthwhile, and it wouldn't matter whether you practiced before or not, you only proceed if you know. This is how real games/school works anyway, if you can't employ all the skills you learnt before, the level boss(teacher) will come and kick your derriere (fail you).

4 years ago

https://www.duolingo.com/Learnerplates

I also find the decay rate quite a mystery, and particularly when you compare the App with the desktop version. I went through a few days where I had to do more on App than on the desktop - i use the App when I have no other option as I find it very limited.

When I was able to get back onto the desktop I discovered a lot of words that were down to one gold bar - and yet I was doing the same number of lessons as before. I couldn't work out whether this was because of my use of the App (i.e. are lessons on the App of less value as they often easier) or something to do with the overall pattern of my progress.

In any case, I am now struggling to catch up with words that had, apparently, suddenly and dramatically decayed. Basically I now have to put in more hours than ever before to keep at the same practice level (i try not to let words go below "time to practice" if possible).

Whilst Duolingo has given me a lot, it has also confused me as to how to establish a regular learning pattern - ironically I thought this would be its strength considering its reputation.

4 years ago

https://www.duolingo.com/Bigface

Hey, great job guys with all the work going on! I was just curious, though: Why not have the strength of words go down based on the number of times a user errors with them (or peeks, etc)? That way it's directly related to what the user may or may not be getting rather than an arbitrary time. With time, it's either too long (and words never decay) or it's too short (and words are decaying too fast for people to catch up). Maybe it's already doing that, but I'd say a combination of time and user error would be pretty useful.

4 years ago

https://www.duolingo.com/tatou
tatouPlus
  • 19
  • 11
  • 8
  • 2

This already happens, but errors do not influence the strength bars themselves, but how quickly they decay. (They get bumped back up to 4 bars whenever you get it correct.) So more errors mean the sooner you need to practice.

4 years ago

https://www.duolingo.com/Bigface

Ah, very interesting. Now, sometimes I'll be able to figure out a word that I'm unfamiliar with because the lesson makes it easier (take the multiple choice sections for example). Will getting a word correct in that situation move my strength all the way up to 4 bars?

4 years ago