Notice some changes in word strength?
We made a change to how word strength is calculated this week.
This fixes a bug that was causing words you know "pretty well" to be "locked in" at four bars, usually over-estimating how well you know them. This also meant that you could never practice those words again, which caused a lot of problems for some of you (like repetitive practice sessions because only a few words were not "locked in" at full strength).
The Good News. This new fix lets you practice words that you previously couldn't, and will lead to a much better learning experience! :)
The Bad News. Some of you (especially those who have been using Duolingo for several months) might suddenly see fewer gold skills on your tree. :(
I wanted to write here to clarify some of the questions we've been getting about this...
Practice tip! To get skills gold again faster, try the "Practice skill" button on those particular skills' pages, instead of "Practice all skills" from your home page.
You do not need to re-do lessons. Unless there is a blue notebook with "+1" in the corner, you have "learned" that skill. To get it back to four bars all you need to do is practice.
The drop in word strength isn't permanent. It's because there are some words you haven't been able to practice for months due to the bug! You will need to do some practice sessions to get them gold again. Maybe several. But it will be business as usual after that.
The more you practice a word, the slower it decays. Some of the suddenly-weak words are basics that intermediate/advanced learners know well, which can be frustrating. The reason is that you have maybe seen them only 4-5 times on Duolingo (and and it's been several months), so the website is a little confused. When you get them correct, though, it should be a long time before Duolingo thinks you need to practice them again.
Many words get strengthened per session. When you practice, Duolingo targets your weakest words and gives you sentences for them. However, all words you see in those sentences are strengthened (not just the 9 or so we show at the end of the session). For example, if you practice the "Travel" skill, any words from "Basics" in those sentences will also get strengthened. (Note: we are investigating a bug in the mobile apps that causes some words to not be updated.)
Timed vs. un-timed practice. In timed practice, you get 1 point for each correct challenge, and all words are updated. In un-timed (and mobile app) practice, you get 10 points + a heart bonus, just like lessons. Also like lessons, none of your words are strengthened if you fail (run out of hearts).
Word strength vs. decay rate. When you get words wrong, they might still be strengthened, because you just saw them and they are fresh in your memory. But they will decay faster than if you got them right, so Duolingo will get you practice them again sooner.
We have been testing this new change with a fraction of you over the past two weeks, and the reaction has been positive. In the long run, people actually need to do fewer practices! But In the flurry to fix this bug for everyone, we didn't do a good job of communicating the change, and I apologize for that.
If you have any other questions about word strength or practice please ask them below, and I'll do my best to answer them!
71 Comments This discussion is locked.
One thing that I noticed is that the "Practice all skills" FINALLY WORKS!!!!! I've encountered words that I haven't encountered for a loooooooooooooong time (some of them I think I've only encountered once)
This is the change I've been waiting for a very long time and yesterday I did something like 20 lessons plus 10 today, experiencing "new" words everytime.
Kudos to you Duolingo developers, you made a good work!
Can you look into an issue with words not being strengthened when practiced in an android app? I practiced several skills in the app and usually strengthened 9-11 words but in my vocabulary it registered only 2 -5 of them per session (and in the app the strength didn't go up). Btw thanks again for this bug fix. :)
Actually, we're investigating this right now! Both the iPhone and Android apps seem to be doing something wonky when updating words (but that a different issue than this bug fix).
Note that in the mobile apps, if you fail a practice your words do not get strengthened (they are more like lessons). This is also true of "un-timed practice." If you do a "timed practice" on the website, though, all the words get updated.
This might not be new information on the issue or even related at all, but in my experience it happens mostly with difficulty syncing between offline and online mode. Even if the lessons are done in online mode, if I go into offline mode soonish after doing a session in online mode then, there seems to be some issue.
Take this with a grain of salt because I have fairly few issues with syncing using the iPhone app.
P.S. I really like the changes to how word strength is done, now it seems much more accurate to which words I actually need practice with. It felt odd before because it routinely said I knew words I didn't know that well and said I didn't know word that I was pretty comfortable using.
I also want to offer some evidence that disputes that the practice skill button is the fastest way to improve skills.
In Portuguese, Education skill, there are 7 lessons. My strengths for the 7 lessons were 3 4 3 3 3 2 3 (where 4 is full strength). There are 6 lessons which are not at full strength and I can get all of these to 4 by doing each lesson, so presumably I could also make all of them at strength 4 by using the "practice skill" button 6 or less times. This is the progression of the scores as I used the "practice skill" button:
3 4 3 3 3 3 3
3 4 3 3 4 3 3
3 4 3 3 4 4 3
3 4 3 3 4 4 3
3 4 4 3 4 4 3
3 4 4 3 4 4 3
4 4 4 3 4 4 3
4 4 4 4 4 4 3
4 4 4 4 4 4 3
4 4 4 4 4 4 3
4 4 4 4 4 4 3
As you can see I gave up after seeing no change in the last 3 times (despite no questions wrong). This is 11 attempts with the "practice skill" button and it still has not achieved what I could have done by doing each of the 6 lessons once.
Thanks for doing this experiment! There might be cases where redoing a lesson is faster than practice, but the way things work that should be very rare. Here is why. Practice strengthens 9+ of your weakest words per session (if you do not run out of time or hearts), whereas redo-ing a lesson strengthens 7+ words (not necessarily your weakest). Were all of these timed practices where you beat the clock?
My guess is that you got the skill as whole up to 4 bars after the 4th practice (the first "3 4 4 3 4 4 3"). So even though not all the lessons were at four bars, the skill was (on average). I also looked at the log data, and it looks like you refreshed 23 words in "Education" over the past 3 hours. That should take about 3 completed practice sessions (without running out of time or hearts).
If we used "Practice all skills" would our skills eventually get back to gold, or does it not influence them at all?
Yes, but "Practice all skills" might help you slog through other weaker words throughout your entire vocab first! :) Let's take an example. At the moment "Directions" is one of your 3-bar skills: 10 of these words are not at 4 bars. If you completed a skill practice, at least 9 of these 10 would be refreshed, assuredly bringing the skill back to 4 bars. You might have to redo 2-3 of the 4 lessons to cover the same ground since the weak words are spread out among the lessons. And if you clicked "Practice all skills," none of these 10 would be targeted because you have weaker words spread out among the other skills (Medical, Household, Places, etc.).
Note: There is a possibility that things still aren't updating correctly for you in practice sessions. If you completed each of these practice sessions and still had this behavior, there's probably a bug. Let me know and we'll look into it further!
I'm still confused as to why it took so many practice sessions to get the improvement I saw. Presumably a lesson is not at 4 bars because there is some words that it contains that need refreshing. Doesn't this suggest that I never encountered those words in the practice? I'm almost certain there is a bug and here is why. When I look at the vocabulary, in particular by Education skill amongst the worst words are:
geografia : 2 bars
introdução : 1 bar
tema : 2:bars
faculdade : 2 bars
I've already mentioned that geografia and introdução were tested many (many many many) times, and I saw tema and faculdade at least once per set of questions. There is no reason why they should have the score as low as they do. Furthermore it says
1) that I last saw these words 2 weeks ago
2) that I've only seen these words twice
This is obviously not correct.
PS. None of the practice sessions were timed, they were just 20 questions/3 hearts/no timer. All of these practice sessions were completed successfully.
OK yes, that definitely sounds like a bug. I've been adding some new infrastructure this week to help us track these sorts of mistakes and figure out what's causing them, so we hope to figure it out and fix this very soon.
Thanks for your patience and for all the info! Two more quick questions:
- Were these problem words (geografia, tema, etc.) highlighted in blue during your practice sessions or show up in the "you strengthened..." slide at the end?
- Were these practices on the website or the iPhone/Android app?
I was using the android app. I can't remember seeing any highlighted blue (but I'm not sure I recall any highlighted blue words on the android app ever). I also don't pay much attention to the "you strengthened" list. Since this is not very helpful I did the following
1) Did another "practice skill" set for Education on Android App
Result: No words in blue. The words that I repeatedly saw such as geografia, tema etc were in the " you strengthened" list and had 4 bars. I checked the vocab section on the website again and it had not changed (ie these words had a low score, not seen in weeks).
2) I then did the "practice skill" set (untimed) for Education using the website on my laptop.
Result: These offending words were sometimes highlighted in blue (maybe it depends on the question type). But yes in particular geografia, faculdade, tema were highlighted in blue at least once. They were all present is the "you strenghtened" list and had 4 bars. Checking the vocab section of the website again and these words have now been strengthened and it says they were last seen a couple of minutes ago. That last lesson in Education that just refused to go to 4 bars now has been fully strengthened.
So... for SOME words the android app is failing to update the word strength but the website succeeds; I have no idea why these particular words are special.
PS I still encountered the problem where the app is giving me 5 times as many points as it should with this practice I just did. (I brought this problem up in an earlier discussion and Luis is aware of it, but I think it may have only been partially fixed).
This is in response to your most recent comment advising that you believe the bug is fixed and asking for some tests. I couldn't reply directly to it since there's a limit on how deeply comments can nest with this system, so I'm putting it here. I just did a practice session with the Android app, and the following words were practiced (Spanish): además, haces, preso, definativamente, piso, setenta, absoluto, dirección, cabeza. I answered all questions correctly and got 13 points for completing the exercise, however it only strengthened two: piso, absoluto.
This is also happening to me at the moment with the "sizes" skill in Portuguese. I used "practice skill" (on android) and then checked the vocab. There are several words which were tested but it says I have not seen them in a couple of months.
I then did a "sizes" lesson directly (on android), and checked the vocab and these words had been updated.
This suggests the problem lies with the "practice skill", and not with the app in general.
I just finished another "practice all skills" on the android app. The words on the strengthened list at the end were all strengthened according to the vocab section so that it is good. However with respect to some other comments made previously:
>> " Practice strengthens 9+ of your weakest words per session"
There were only 8 on the list
>> all words you see in those sentences are strengthened (not just the 9 or so we show at the end of the session).
One sentence that appeared in the exercise was "Os jovens que a gente tem encontrado não entendem o estudo". Checking the vocab section I can see that "encontrar" was strengthened, but it says I have not seen "jovens", "gente", "entender", or "estudo" for quite some time. Perhaps there is another bug?
OK, this is another issue we are also working on. It's because "entendem" is the third person plural present indicative form of the verb "entender." The system did update that you saw this word, but not the general idea of the verb "encontrar" (which is what we use for the "last seen" time on your vocab page). My guess is that Duolingo was trying to get you to practice "encontrado/encontrar" so it knew how to update both the surface form and the general word for this one, but not the other words.
We are working on this, but unfortunately it involves a non-trivial infrastructure change in the software so it will probably take a bit longer...
Yes, it turns out there is a bug in practice specific to Android, which we are working on now. This should be fixed in the next few days, and hopefully it won't require an app update for you. But we will make an announcement when it is.
In the meantime, practice on the website will work better. Sorry for the trouble, and thanks for helping us figure this out...
Also, the same words kept reappearing and reappearing... For example, out of the 20 questions seen in one particular "practice skill" question set, the words [geography, introduction, education, test, discipline] each appeared twice, and not in a complex sentence either - half of the 20 questions were asking for the straight translation of these 5 words. A further 6 questions focused on the words knowledge and intelligence, but sometimes used them in a complex sentence. In the last hour I have done 11 "practice skill" sessions and have probably translated these 5 words over 20 times each. I never want to see these words again!!! lol
I've encountered this as well, in fact on the practice session I just completed, 1/3 of the targeted words were the same phrases repeated over and over (different methods, i.e. english=>spanish, spanish=>english, muliple choice, etc. but the same underlying sentence) I think this goes a long way towards explaining something I mentioned in the other thread. These words / forms degraded while this bug was in place specifically because they're not likely to be used in practice exercises unless the application is specifically targeting them for improvement (which wasn't happening before this week). The fact that there are so few sentences available for some of these words seems to be why these 500+ newly degraded words (for me) will require dozens and dozens of practice sessions to fix.
As a follow up to the slight controversy regarding your statement in the other thread that "a few practice sessions" should get things back in order, it's been a busy week for me, and I've only had time to do 15 or so practice sessions since the change (no new material at all). I've improved 135 words or so, and still have several hundred words to restrengthen. Most of these are words that I know. They're minor variations of words that are at full strength, and this exercise is tedious, annoying and thoroughly frustrating. Surely I'm not the only one in this position. I saw another user mention that he/she had to do ~100 practice sessions to get things back to normal - surely that's not a burden you intended to put on your users.
I mention that only to set up this next question: Would you guys consider, as I mentioned in the other thread, modifying the existing "test out" method of skipping skills that we know to allow for quicker skill refreshes? I know you're a relatively small team and developer resources are finite, but this doesn't seem like something that would take a lot of effort to adapt to this purpose. Given the impact that this change has had on some of us, I think it's a reasonable request, and one that I hope will be considered.
Edit to add: Using my own situation as an example, to get everything back to full strength, I can delete all of my Spanish progress, do 3 30 question shortcut tests, and 6 20 question "test out" exercises to completely get everything back to full strength, or I can do roughly 65-70 more practice sessions to top off my existing skills.
One thing I just noticed is that even though I finished the French tree months ago, and since then, when they added new lessons, it seems the golden owl at the bottom of my tree disappeared. Does this mean that whenever new lessons are added, Duolingo effectively takes away the accomplishment of finishing your tree?
I lived in France for a year, and I consider myself to be flawed but essentially fluent. While it's nice to do the speed practice as a personal challenge, it's kind of annoying to think that I have to keep going back to lessons when what I really want, (and what I suspect Duolingo wants from everyone once they've attained proficiency) is to do more translations than lessons. While my translations can certainly improve, I find that the lessons are annoying and time-consuming if I'm not really learning anything new except which particular wordings Duolingo will accept.
While I do understand the benefit of decaying word strength and review, I really enjoyed the sense of completion with the tree, and I'm somewhat annoyed that it's gone now. I would much prefer it if words from translation were included in word strength on one's tree as well. Plus, I think there should be a moment when you can say, there, I did the tree, that part is done, now I'm going into full translation mode.
You are right, taking away the golden owl which is a sign of your accomplishment is annoying and unfair ! Just imagine you would have to renew your driver´s license if a new car goes to the market, or you have to redo all your exams again if they became void with a new scientific development :-((
And while doing translations increases the number of skill points you have, it should also be added to the number of words you have learnt; I have not idea how the programmers can do that, but somewhere somehow within the structure of the owl's geeks there is probably someone who does.
Maybe after earning your golden owl the relation of repeating vocabulary and points and words earned from translations can change, say 1rd for voc and 2 rd for translation, plus the voc from translation added to your word list ? There is always hope... :-)
This is not my experience. I still find (1) that I still lose bars (in German exercises) too fast, so my progress in getting back to gold is very slow, almost backwards; (2) sometimes I see the same words over and over without seeming to get credit for them; and (3) many of my "peeks" are accidental (the pointer accidentally moves across a word), so I would not like to be penalized for them. A consequence of this is that I don't take the time to translate web sentences, though in the long run that would probably contribute more to improving my skill with the language.
I've been working hard the last few weeks in an attempt to try to get all my words up to full strength. As some background I have currently 1230 learned words (all lessons currently and constantly gold) and approximately 450 of them were decayed around a month ago. Since then I have practiced (mostly using strengthen skills from the web page) every day for a minimum of 50XP points. Checking around a week ago to see how I was progressing I was dismayed to see that I had made zero progress in refreshing my word list.
Thinking perhaps that words were decaying at the same rate that I am refreshing them spent the last week doing significantly more practice (500-1500XP a day). I've been keeping track of the affect on my word list after each 100XP of practice and I can see that typically five practice sessions might only refresh one or two words? It seems like the words being selected for practice are not the words that are listed as needing refreshing? It's very demoralizing to feel unable to make progress no matter how much effort I put in ... the last week is an unsustainable amount of effort anyway.
I could try and ignore the issue and continue progressing with new material, but I report this in the hope that it is helpful in identifying any bugs or aiding in the development of your learning algorithms.
See this URL for my small analysis of the subject. https://www.duolingo.com/comment/9917033
It seems focusing on weak skills works better, if you make dumb mistakes words can be set back from strength 4 to 1 and if you peek the strength will decay faster.
Look through the word list and check how well you know them. If there are still many difficult words in the list you can practice that skill (click the word to find the skill). You can even consider to practice all skills from a certain starting point. In that case keep practicing each skill until you know it well enough. You'll have to keep note yourself at what skill you are.
You can also try if learning a new skill is not too difficult. If you don't stumble upon too many old words that are still difficult for you, just ignore the weak strength and move on or strengthen the skill that made you stumble.
Pasting the whole word list into Excel made analyzing it a lot easier for me but it sucks that we have to resort to such methods.
I decided to look a little more closely after having persevered with trying to brute force get all my word strengths up, and after reading NicoLaan69's similar investigations. After doing more than 10,000 XP worth of "strengthen skills" practice in the last couple of weeks it was clear that I was only practicing words that were already strong, I was rarely strengthening any of the words that are in need of practicing.
My first thought was that there was a problem with algorithm that determined which words to present to me. I then tried strengthening skills (or redoing lessons) specifically for skills containing weak words. This worked in some cases but in a large number of cases no matter how many times I tried the weak word still didn't come up. "Haces" is an example of a word in this category for me. I also noticed as I looked more carefully, that some other words ("una" is an example) don't get strengthened, even if it does come up during practice.
I hope someone takes a look more closely at this, there seems to be something significantly wrong in the current implementation
Thanks for telling us about the changes in practising word strength. I understand why you changed the practise system.
But Duolingo did not take into account that some people have finished certain language streams already (and got rewarded with a gold trophy :-)) as I did in Spanish.
After my having finished Spanish und the implementation of the new practise scheme, Duolingo has added NEW words to the different streams, and in spite of practising these NEW words, these NEW words are NOT ADDED the total number of words learned
Can you check that out and adapt the word count accordingly? Thanks
Unfortunately didn't see any words or skills decaying on my tree. :-( Though I love the idea I am practicing everything and enough, I guess it's not the case. Can it be that some accounts still have that bug? Just to clarify in my vocabulary there are some words having full golden bars with indication that last time I've seen them was over a month ago.
Thank you again for the great job you are doing! Can't stop recommending this website.
Is it possible that this update "accidentally" removed the COLORS lesson from the Italian language tree? It was there a few days ago before I unlocked it. I finished the first section of the tree about 3 days ago, and it was there, but I stopped at that point (I had done like 12 lessons). I went in early yesterday morning with the full intention of doing the COLORS lesson, but it was no longer there. It prevents me from progressing any further, because the next block available requires the COLORS lesson to be completed as a prerequisite.
I think there's a problem in Duolingo's report of achievements. I've been repeating many lessons that I have already cleared before and they just don't recovered while I practice. It's definitely frustrating and discouraging. (Besides synonyms lacking for portuguese lessons).
Are you repeating the lessons because you're trying to keep the tree golden? You don't have to keep it golden to progress- that's one way of going about it, but it would take forever to finish a tree if you did it that way. Either that or you would have to have hours every day to devote to maintaining your trees.
I can see that you are doing multiple trees, so you probably will be very frustrated if your goal is to keep all trees golden as well as finish the tree. I think keeping the tree gold is a good goal if you're only working on one language or have lots of time to spend, but way too frustrating if you're working on more than one. If you progress, you will automatically refresh some of the skills from before, but some of the skills will definitely decay a bit.
The tree being gold or not is only a method to return to items that need extra practice but IMHO it's flawed. When a skill is new I find that I need practice when it's still gold and when it's old I don't need extra practice when DL says I do.
Tip: Check the words in a skill, look and consider if you know them well enough and practice based on your own judgement.
If you go speeding down your tree, learning new skills very fast, you'll likely get stuck because you don't know the new skills well enough. So for me I practice the new skill even the same day I learned it and the next day and the next until I feel I know it, after that I'll have to trust DL to warn me for extra practice. You get less XP's this way, but the object of the game is to learn a language NOT gain XP's.
Note that there are different kinds of practice. On individual skill pages, there is a "Practice skill" button that will target that skill specifically. The "Lesson practice" button on the home page tries to make you practice the individual words/concepts you need most, but won't necessarily strengthen a particular skill, because the words come from many different skills.
As a rule of thumb, I recommend doing 1-2 new lessons and 2-4 practices (skill or overall) every day, depending on how much time you have. Over time your skills will become gold again, but you shouldn't let decayed skills keep you from moving forward.
Whether I re-do lessons or let DuoLingo choose words for me to practice, the "German words learned" list doesn't always seem to reflect my efforts. Some words do show up as last practiced recently, while others seem to remain stuck at "last practiced 2 years ago".
I have recently gone through my German skills tree and practiced until everything became gold, but I might have missed some individual words (for example, I notice each inflected form seems to be a different "word" in the list).
So I went back to the specific lessons where these words are introduced and used those to review, but in many cases there was still no change in "last practiced" date.
Now I am just letting Duo decide what I should practice, which is fine, but there are still words such as "Auto", "Computer", "Hilfe", "Mittwoch", "Mutter", "Lehrer", "in" and "wurden" which show up as last practiced two years ago even though I am sure I have practiced them recently.
So I am wondering why some words seem not to update, while others do. The ones that do update sometimes show only one or two bars, which is fair enough, but at least the "last practiced" date is correct for those.
I've kept my Spanish tree gold for more than a year now by doing a few newly colored skills a day. However, I notice that more than half of my words in the word page are overdue. Any recommendations on the most efficient way to update words in the overdue category while keeping my tree gold? Thanks, Steve
Difficult and time-consuming. Practice the skills that need practice the most. It seems a general practice doesn't help much if at all. The major problem if you decide to do this, finding the skills that need practice the most! I managed to do this (July 2015, 70% of the tree done) using Excel and the word list.
1 Take the word list and sort by strength
2 Paste in Excel, copy-paste from Excel into a new area in Excel (paste as values, I don't want Duo's formatting)
3 Enter the strength yourself, because copy-paste does not copy the strength (it's sorted on strength so this is easy, find the words on the transition between 1 and 2, 2 and 3, and 3 and 4)
4 Find for words with the lowest strength the skill they belong to (click word in Duo and look in the small info box on the right
5 Create a small list with only all skills in it and the number of words with strength 1 through 4.
6 Based on this list you can easily see what skills need practice most.
I told you it's time-consuming. :-/
I've an Excel sheet with almost 1000 words and their skill.
Clean Excel sheet (no macros) https://drive.google.com/open?id=0Bw28fiT93rt0WmdHdmZIeTRYOTA The macros if you want them https://drive.google.com/open?id=1zfOrkRxEjurxgNcOExov9o-qXUTr1bVp7ieIgCZRgK8
I suggest a download, because google docs seems to find it difficult. Step 2, the "paste as values" can be done in sheet "FindSkills" cells E3. Sheets 16, 17 and 18 can be used to compare your progress on different dates. As you can see, I've taken a holiday from Duolingo of some months hoping they will improve things.