1. Forum
  2. >
  3. Topic: Duolingo
  4. >
  5. Update on Skill Strengths


Update on Skill Strengths

Hi, I’m Joseph from the learning team. Our role at Duolingo is to continuously improve the learning outcomes of our learners.

Many of you have noticed a shift in your skill strengths, and I’m here to give a brief explanation of why this happened.

As you may know, our team constantly runs A/B tests to try new ideas and measure their impact on learning outcomes. (An A/B test is a controlled experiment that splits our learners into “A” or “B” groups, each with a slightly different experience.)

One recent test A/B test boosted word strengths for a group of learners. We hypothesized that our existing strengths under-valued learner proficiency, and that increasing the word strength would have a positive impact on learning outcomes.

The data shows this was not the case. In fact, learners who were shown boosted word strengths performed worse than learners who were not shown boosted word strengths. So unfortunately, those of you who were in the experiment will see your skill strengths returned back to the level of all learners.

Even with this explanation, we understand how frustrating this will be for those impacted. We always aim to implement changes that lead to better learning outcomes, and this time that was not the case.

We, like you, are continuously learning!

April 18, 2017



Thx for the update and explanation!



  1. Do you confirm this impacts "Words strength", "Goldening" and "Fluency"?
  2. Does it impact something else that us users could notice?


You're welcome!

  1. Yes, this impacts words strength, goldening and fluency.
  2. This will also impact what words will be reviewed in a practice session.


Thx for the answer!


How about the opposite effect? I was never above 48% fluency for German. Now I'm at 59% !!! My Spanish went from 56% down to 51% (even though all golden and really my 2nd best reading/understanding language), and Italian from 52 or 53 to 48%.


My German fluency (L14) increased from 28% to 39%, and I've hardly done any German lessons in the last month ??

My Spanish fluency (L19) dropped from 55% to 50%; My Portuguese fluency (L19) dropped from 53% to 47%; My French fluency (L17) increased from 53% to 54%; My Italian fluency (L16) dropped from 47% to 40%;


This is interesting because my German fluency ALSO increased (from 50% to 62%). It seems that German fluency improved while other languages dropped. I wonder what is special about the German tree...


I'd been wondering that myself, but figured they just might have fixed whatever had kept my German "fluency" numbers in the 30s for many months, although I've been studying the language for decades.

Recently my "fluency" score in German jumped by ten points or so -- while my score for Spanish, which I've been studying even longer than German (and have more conversational fluency in), DROPPED by the same amount. Go figure. :/


Spanish fluency makes no sense since the "a/b test". I was dropped to 34%. I have completed many new skills, jumped two levels, even completed a recent "surprise" 5 question fluency test, which I think I answered correctly, I am now 5 lessons away from completing the tree and have been keeping the tree golden, yet, I still sit at 34%. What gives DL???


Fluency's about more than how much of the course you've completed. It also takes into account how secure that knowledge is. Broadly speaking, if your tree is gold, your fluency percentage will be higher than if it isn't. The longer the tree has been gold, the higher the percentage will be.

There's some more info here: https://www.duolingo.com/comment/19032758


Yes, I have read those posts and that's why I am saying in my case it seems to be off based on the criterion given. I have now completed the tree, I completed vocab 3's 10 lessons without any errors I think the first time and I'm at 33% now. I think they are way over rating incorrect answers as I normally get through strengthening correctly or 1 sometimes 2 incorrect answers. I do about 4-8 lessons each day to keep the tree golden. It doesn't make sense that I am at about 50% of the 60% max achievable score. Anyway, I will continue to strengthen going forward and it will be interesting to see what happens.


It's good to know that this is a one-off (at least until the next experiment...) but is there any experimental data about the learning impact of suddenly finding you have lots more to do than you thought? Is there any way to phase in a change like this so that e.g. the learners in the boosted group find that they have a few more refresh lessons to do each day until they have caught up rather than seeing 50+ all in one go? Also it would probably be worth making this post sticky at least temporarily to reduce confusion.


And now my trees are all/mostly gold again! So what does that mean and what will the morning bring?


Just refreshed my page and suddenly all the skills regilded. Annoying that I wasted time this morning redoing about 9 or 10 skills which it seems would not have needed doing had I left until after midnight! But, like you, wondering what tomorrow brings?


Thxs for clarifying the discussion section has blown up with questions about this


OK, thanks for the info! I belong to the group that was put back down. I was not successful in keeping my trees golden before, so it will be close to impossible for me now. Almost all my skills in my German tree are down to one bar now.

(Who is giving lingots to the staff? FYI they can give themselves unlimited numbers of lingots by editing the database. Lingots are of no value to them.)


Have you tried buying timed practice in the web version of Duolingo? I found it really useful when my skills first started fading after testing out of the French, German, Spanish and Italian trees. Until yesterday all four of those trees were golden. I guess I’ll have to first gild all the skills before testing, but I think it shouldn’t be too difficult to keep all my skills golden, even with this change.


Haha, I almost had a heart attack when I woke up and saw that 31 of the 47 skills I have need strengthening. I'm glad it's a one time thing, I thought I just hadn't practiced enough.


You've got to be kidding. With key investors such as Fred Wilson in the Duolingo platform, I would think you would understand how crucial your communications are with your base. Transparency is expected and would certainly help your KPI's while you are figuring out your revenue model. Trust is crucial with users. You need to develop a better customer communication channel.


For some experiments warning the subjects is ok but for others it can affect the results and thus isn't advisable. For this specific case, if you want a thing to complain about it should rather be the "boom, reset!" action, it would have been better to apply the change slowly over time so people don't wake up to 'all' skins not being gilded anymore.


Just downgraded from 54% to 46%. I understand Duolingo tested some ways to improve its evaluation system. But, please, before applying your exits just put a warning so that we are all ready to know and to act consequentely


I have the opposite problem - about 2/3, maybe even 3/4 off my words in my Spanish (from English) tree are shown as less than full strength, mostly with only 1 bar, but almost all of my skills are shown as golden - this is after several weeks of neglecting that tree while I work on the reverse tree. (When I left it, all the skills were golden and staying golden, despite about 1/2 of my words being at 1 bar - and despite a huge amount of hard work to try - but fail to strengthen them). I know this will sound like luxury to some - but it's actually really disheartening! Do you have any explanation for me??


There have been various discussions about inability to get words to full strength. Agreed it would be great to learn more about what's going on!


Install the user script "Skill strength viewer" and check the "Dead words count"


For me fluency percentage is interesting to see, but not especially important.

But: I would like to know why, after completing all of the lessons for a given language, we can only refresh it by laboriously completing lessons that are very easy but still take time. Why aren't we allowed to test out again the way we are when we first complete the lessons? It does not make any sense to me.

I know that if I just concentrated on one or two languages, it would be easier to keep them fully refreshed all the time. But one of the things that makes Duolingo unique is that it allows language learners to study a variety of languages.

I don't expect things to be set up just to accommodate those of us who are learning many languages, but I don't understand why it's OK to test out of lessons the first time through, but not in subsequent sessions.

Is there a reason?


For nearly 2 years I have been a daily Duolingo user and have really enjoyed the experience. Then, last month my 575 day streak was unilaterally aborted... unless I paid $9.99 to restore the streak. Strike 1. Now, we learn we have been used in an A/B test and our (both mine and my wife's) fluency levels have returned to a level not seen since November 2015. Strike 2. Duolingo has changed into a clunky app that I no longer can recommend to others. There is no longer the consistency and progress milestones that we previously looked forward to achieving. Trust has been casually abused. How can we know if anything we do in the future on Duolingo has any longevity? Sounds like I am not the only one feeling intellectually abused...


winecuador you're correct. My friend lost her 100+ day streak even with a streak freeze and no help from Duo other than she was offered the same $9.99. I use to LOVE Duo and recommend it to everyone. No recommendations until they get their act together.


The result of this experiment was not understanding what is happening to me, a desease or something as it seemed that I forgot the Basics1 in Italian, at the time when I was trying to learn difficult declensions of verbs in the most difficult part of the Italian tree. Much frustration!!!

Alright make the experiments you want but first give us a feedback what to expect, please.


To tell beforehand would invalidate the result because of the resulting bias, making the test mostly useless.

In this particular case I would say the problem is that they waited too long for an explanation after the switch back. Even better would have been a gradual switchback over the course of about two months so that nobody noticed.


Hopefully I suspected that something is happening, but well, the surprise was too bad, and I spent some hours to make the first levels golden again. The same happened to my French tree that I had done it all golden the previous day and suddenly I found out that about 40 or more subjects were not golden anymore. After a couple of hours suddenly all became normal again. Phewww! End well all well, but please Duolingo do not it again without warning, we will have strokes or heart attacks :)


Perhaps if we could still do flashcards under the Words tab, our skills would have been higher?? Why not give us back that capability??


You don't have the green "Review flashcards" button in the "Words" register on the web?
Then you may want to try the user written Android Flashcards app by lekz for DuoLingo API, announced on the forum: https://www.duolingo.com/comment/19154823
The author said he added the customization on how many words you want to practice (words cout is fixed to 15 on DuoLingo web).


If Duolingo is going to maintain integrity, then transparency is in order otherwise users will leave. After X mistakes by the startup team, users will find alternatives in platforms that have real customer service. Given the history of these problems, the outcome could have been anticipated and dealt with appropriately. If you value users and want growth rather than churn in your statistics, then hear, listen and learn from your customers.


Has the problem not been corrected for you? It has for me. My tree went back to being mostly gold, and I suspect they'll employ the new algorithm in stages rather than all at once. I have yet to find a learning tool as effective as this. I've been trying to buckle down with Italian for 10 years and I truly feel overwhelmed by Rosetta, and not a huge fan of classes. There is a lot of room of expansion on this tool, like on-line tutoring, etc. I'm very happy they reversed the changes yesterday.


Can you incorporate "Skill Strength Viewer" user script to site?


This gives the percentage raw "word" strength of each Skill. It is extremely useful in deciding what skill to work on next when all the tree is golden :)


I guess the most important is the "Dead Words (0 Strength)" besides the bugs:

  • "practiced words stay always 1-2 bar strength"
  • "last practiced" database field is not correctly updated, even the word has definitely been shown/used (shown) in an strengthen excercise.


Yes! it would be great if the Duolingo Team could fix the dead words problem. I try practicing but I never get to strengthen them.


This is really demotivating and lacks transparency!!!


@JRollinson I just noticed two anomalies for my strength/fluency stats. Does the change to strengthen skills fix these:

  1. I use "strengthen skills" regularly, yet the "Words" tab shows 100s of overdue words that haven't been practiced for years. Shouldn't duolingo be showing these words for me to practice?
  2. My fluency is now 61% for German. Isn't this level too high, I thought it maxed out below that?


My tree is all golden as normal, and my German fluency has risen from 50% to 63%. So, I'm thinking this is a good move! ☺


you were probably on the side that it was harder to strengthen skills, so nothing changed for you.


Can I get an explanation why I have been trying to gild one subject with one bar left and it won't gild? I've "strengthened" it 4 times in a row!


That's a separate issue. You've come across a bug that appears from time to time. If you re-do the lessons, the skill will strengthen.


Re-do the lessons as in all of the glitched ones, individually? Yeesh. I'm going to end up going from around 30xp a day to something like 200-250xp just to reguild my tree between the ones that unguilded today and the 3-4 ones that won't reguild unless I re-do them. Especially if I want to finish the tree in the timeframe I wanted to (within the next 4 months).

I wish I had more time in the day...


I agree, this just feels like a disastrous move on DL's part. They should not have implemented this change without addressing the bugs first, because this bug has been around for a bit. Also, they should have implemented this change in 5 or more increments over the period of months, not one day. As I said before, if I weren't so far along, I might just have gone to another app or program.


The ones that have weakened because of the change to the way skill strength is measured: you don't need to re-do all of these lessons, just strengthen them as normal.

If you've encountered the occasional skill that won't strengthen no matter how many times you do the strengthen lesson, then that's when you need to redo the lessons.


You could just put a little Notification icon next to the score such as "What happened to my proficiency level?" etc. I know the numbers don't matter. I do not care about them myself but Duolingo itself is pushing play/reward aspect of the site more than I care for. If they are doing that, they should also be acknowledging that some people would be upset about these arbitrary changes. The changes cannot be anything but arbitrary because you cannot evaluate someone's language proficiency with the amount of information you can gather on Duolingo.


Thanks for letting us all know.

[deactivated user]

    This is disappointing communications by the Duolingo staff. There have been at least 50 discussions posted on this issue just in the last 24 hours, not counting ones that have been downvoted to invisibility.

    The lack of communication on an issue that affected a significant proportion of the Duolingo user base is unfortunate, to both those who are were affected, and users and moderators who have to try to manage so many new discussions on the same topic. It would have been easy enough to make this announcement before you made the change not 24 hours after.

    The forum users who will now discover the reason for the skill degrade are the lucky ones. What about the app users, who don't read the forum (at least 95% of your users). Are they to log on and think the app has glitched? Not great PR when you are about to launch a paid option on the Android app. There are messages whenever a new version of the tree rolls out. Similar ones could be used to let users know of the change.

    I have been supportive to Duolingo over my time here, and want to see this webiste grown and succeed. But incidents like this isn't the way to do it. When you make a change like this, please let us know ahead of time.

    Happy learning :)

    EDIT: As hughcparker pointed out, staff has been much better on making announcements than before. I have noticed that as well, and want to thank you for that. But if you plan on making an announcement anyway, doing it before hand would seem to make more sense, and save everyone a headache.


    I wouldn't say that communication is the main problem here, 90%+ of those affected won't even see the communication most likely. The main issue I would call the "boom, reset!" action, it would have been better to apply the change slowly over time (for example stretched over a month) so people don't wake up to 'all' skills not being gilded anymore. This would have avoided people noticing anything and the need for communication in the first place. A lot of communication about every single A/B test isn't the best thing either as that would constantly disappoint the people who liked the thing that didn't pass the test. Obviously there are also cases where communication is a must - like it kinda was in this case but mostly only because of the hard reset.


    I second this. I've been using Duolingo for the past 3 years and had never been to the website or the forums until the sudden drop in fluency rating happened and I sought out why this was happening. A beforehand notification would've made so much more sense.


    It's a day late, and maybe it's poorly targetted, but it's communication from the Duolingo team. They're moving in the right direction - let's encourage that.

    [deactivated user]

      I agree with that. Communication has certainly improved in the right direction since I joined. But if you plan on making an announcement anyway, doing it before hand would seem to make more sense.


      Yes, I agree completely! While I appreciate all that Duo does, I absolutely detest being used, unknowingly, as a test subject for them. Perhaps this primarily stems from my now-missing activity stream, but I feel this website has lost almost all of the charm it had when I first became a user.

      We should be told ahead of time when they begin making such big changes. As someone who currently works three jobs, it's going to take me a while to catch up to what my trees were before.


      Just one thing "I absolutely detest being used, unknowingly, as a test subject for them". If you really do, you should quit now. Actually, there are tons of A/B testing in Duolingo and we're all test subjects. I know that I am, and even if this kind of inconvenience bothers me, I know that it's hopefully for the best.


      Alternatively, when new testing will be done, users should be informed via email or notification.

      We (=users) would be spammed and it's probable that (almost) everybody would opt-out of being part of A/B tests (if not quit Duolingo for spamming them).
      Duo runs A/B test all the time and on everything(*). For example, there are currently 450+ A/B tests running...

      (*) let's say "almost" everything: I don't work there I can't be sure ;)


      Wouldn't all the different tests at once affect the results? Unless they're all disjoint groups.

      No need of disjoint groups.
      Let's take an example with 2 A/B tests. But it applies the same, just with much more distinct cases of "users being in the same group for all test except for the test number N".

      Any user is in one of the 4 cases:

      • Case1: A of TEST1 and A of TEST2
      • Case2: A of TEST1 and B of TEST2
      • Case3: B of TEST1 and A of TEST2
      • Case4: B of TEST1 and B of TEST2

      So to evaluate feature tested by TEST1, you'll compare average result for {Case1+Case2} with average result for {Case3+Case4} and to evaluate feature tested by TEST2, you'll compare average result for {Case1+Case3} with average result for {Case2+Case4}


      "For example, there are currently 450+ A/B tests running..."

      Wouldn't all the different tests at once affect the results? Unless they're all disjoint groups.


      It's not like they're doing something unethical with your data, or trying out potentially damaging drugs on you.

      All they're doing is trying out slightly different versions of the website on different users, and seeing which works better.

      To put it another way - if there are two different versions of a new feature, which group should feel aggrieved?


      I would hope they're not doing anything bad with the data collected. As a scientist, I know how important it is to have a large sample size for research, and I think it's great to conduct research on what learning methods work best for the majority of users. One issue, however, is that everybody learns differently. While Style A may work for most people, it may be ineffective for others. Another issue is that Duolingo is making abrupt changes due to testing without informing its users. When an individual joins Duolingo, she should either agree to being used for testing or have the option to opt-in/opt-out. Alternatively, when new testing will be done, users should be informed via email or notification. Websites often run into issues and are unavailable for short periods of time or may have certain features crash. This is understandable and can usually be dismissed, especially for websites that are used once in a while. For a website like Duolingo, which is often used daily, these frequent road bumps are frustrating and disruptive to the learning process.

      Many users are upset by recent changes, and I think we all have a right to be. Regardless of the reasons for the changes, individuals cannot control their reactions. and any emotional response is absolutely valid and should not be dismissed.


      This is not an issue they would plan on telling us specially when it was not implemented, they decided to post about it because of those complaints you mentioned. Anyway, I don't think I would make a post on the forum to complain about my skills losing some strength.


      I have been waiting and watching for an update on the Fluency %, it is the #1 wanted fix and is the most powerful encouragement that Duolingo could have, instead I get an owl popping out like a cartoon telling me I'm great or now useless achievements that were added. I hate companies who do not respond to users, Duolingo seems to be doing this. So Duolingo do you want to destroy the great thing you have or give the people what they want? Make the fluency percentage work right, explain to us how its supposed to work please.


      I think the people probably want the Fluency measure to work properly. This was a test to try and improve it; Duolingo have listened to the results of their testing, and rejected the suggested change, because it didn't make things better.

      There's some more info about how the Fluency measure works here: https://www.duolingo.com/comment/19032758


      I practice every day, I speak spanish in real life everyday, I know I'm improving but the % never moves up it only went down with practice. I disagree it does not work and its very discouraging


      Are you on the App or the website? Did you see my other comment?


      @HughCParker, I appreciate your comments and explanations, and you are clearly know a lot more about the inner workings of Duo Lingo than I, but I would really like DL to chime in on this topic.

      I read the link you provided and, according to it, I am (and @WilliamHud8 and other posters are) doing everything DL suggests to improve fluency -- practicing daily, keeping all skills gold, picking up new vocab, and strengthening -- yet, even after the Fluency Fiasco, my fluency has dropped 2%.

      Unfortunately (for me), if you put a metric in front of me, I am going to try to improve it. I can't "just ignore it" as many in the community have suggested. I find it very frustrating that I cannot seem to find the right control knobs to make the fluency start moving in the right direction.

      Again, I would really like DL to chime in on this discussion. Reiterating a comment of mine in another part of this post: tell me what I'm doing wrong or let me turn the darned thing off.

      (P.S. Finding no other way to contact DL directly, I submitted a bug report on this topic last night. If I learn anything from that I will be certain to post it.)


      I'm with you on that - if the measure is there, it's difficult to ignore it.

      I don't think anyone from Duolingo is going to chime in - there aren't very many of them, and they don't appear on the forums very often.

      I think the most important thing to realise is that it takes a long time and a lot of work for the fluency percentage to move. Keep all the skills gold for a few weeks, and see what happens.


      Yeah I have too, I admit I have not mastered all the Spanish yet, something people seem to misunderstand, finishing the tree does not mean you are "Golden" but practice, practice and practice till you mastered the Duolingo tree is where one should concentrate. I still do and will.


      I am not an experienced computer site user. A site map would be helpful.


      For some reason a bunch of skills that had been unstrengthened became strengthened to gold.


      Crazy psicological games you are playing here guys. 1st: Hello! 43 skills need to be improved today. 2nd: Geeeez let me start working on it. 3rd: This is ridiculous. Let me Google this. 4th: I see people complaining about this three years ago. Is there any hope? Should I just quit? 5th: Oh! I should go to another discussion, says here. 6th: now people say it has been corrected. 7th: suddenly my tree looks logical now. 8th: Duo, please stop playing with my feelings this way. You can't always glue back together a broken heart.


      I feel like I just found out I was in the placebo group for a life-saving drug! -_-


      It doesn't matter to me where the percentage is as long as I'm learning and increasing my knowledge and use of the language. Thank you for your honest and transparency!


      Not letting people know they're a part of an A/B test also eventually results in user bias, especially when Duolingo turns metrics and data into something arbitrary. How do I know if I'm actually improving my score or again part of a new test? All Duolingo's feedback becomes susceptible to doubt. Two days ago, my fluency score was at 58%, now it's dropped to 51%. But why would I believe that 51% isn't just the score given by another A/B test and that in a few months I'll be told that the "real" number is 38% or 61%, or that if my fluency score improves that that too isn't only the result of a new A/B test? Duolingo's implementation of its A/B tests makes for worse learning.


      It´s very frustrating to loose 15% over night with a fully (!) completed tree. Thank you for your motivation.


      You need to distinguish between new language learners and those with decent fluency who are using duolingo to keep from getting rusty.

      As it is now, learner proficiency is overvalued for new learners and undervalued for those who are somewhat fluent.

      It's easy to distinguish between the two: create a scale base it on how many levels the user tests out of.

      It's frustrating how quickly things de-gold for me in Spanish (it started within a day, iirc!) despite testing out of the tree completely on the first try. I don't need to practice it nearly as much as, for example, Swedish (new to me), which keeps things gold a surprisingly long time, even if I stop practicing it.

      Testing out should cause the program to assume at least as much pre-practice as it would take for someone to reach the end of the tree from the start.

      ...and since you (duolingo team) like data, here's some information on that: I've been making more errors in Spanish recently out of pure frustration and impatience, not for any lack of knowledge. They come from typos and/or reading too fast (ie not paying enough attention because I'm bored and just wanted to get my entire tree gold again).

      Also, I'm no longer going to recommend duolingo to anyone who wants to keep their skills up in a language they already know (unless you fix this). It's far too frustrating at this point.


      I agree. If I'm going to spend a lot of time with the basics, I'd rather do it with a new language instead of repeating the same items I've already completed. So I have neglected the trees for languages which I've been studying for a long time because at this point, there's not a lot of new learning going on by constantly refreshing the tree.

      I have been pleased to see new lessons being added to Spanish and German (I haven't looked at the other trees to see if the same is happening there), but the new vocabulary is still pretty basic and there's nothing new as far as grammar. Now that Immersion is gone, I feel that it's easy to hit a ceiling in Duolingo and not progress much further. Immersion was a huge help for me in acquiring new vocabulary and improving my ability to deal with sentences that are more complex.

      Instead of just repeating the lessons we've already done, I'd like to see new vocabulary and new grammar topics, and I'd still like the option to test out even if we have completed a tree. One of the advantages of attempting to test out is that you're on your own, because there are no hints. To me, that is a more difficult form of practice and review, because it really shows you what you know and what you don't know. How many times do I need to repeat "El niño bebe leche" before Duo accepts that I really know it?

      The main advantage to the current system is that I've been motivated to learn many other languages rather than spend time reviewing material that I know I have already mastered. So indirectly, it has caused me to tackle languages that I might not have tried otherwise, but I'm pretty sure that is not the intent.


      I love Duolingo, don't get me wrong, but some sort of heads up would have been excellent a few hours or days prior as it sounds like this was a specifically planned move. I would have loved to get this message when I turned on the app yesterday so I would be mentally prepared for this. I was clearly in the group with boosted word strength, and had 50+ pods "ungilded." That was just intense for me, after devoting so much time to maintain the tree every single day for 177 days! I agree with Teenage_Polygot this has to be a massive blow to the user base. If I weren't so far along, I might have switched to something else, instead I'm going to buckle down and try to catch up with these 50 pods. On top of everything, it didn't (still doesn't totally) make sense why I have to do the now "ungilded" pods so many times to get it to turn gold again, when the strength bar is one notch short. Would love more clarification about this. I didn't know about these forums until 2 weeks ago. I am very thankful for Duolingo, but this wasn't a well executed move.


      In the end, I think user satisfaction had to be the top priority. I really don't care what the app says about my fluency level, I just like feeling like I can accomplish a few lessons a day, keep the language strong in my brain, and not feel overwhelmed by an avalanche of lessons. I meet once a week with an Italian tutor and she feels in the blanks. This app is great for practice and familiarization.


      I totally agree with all of Bobbya1's comments here. I was shocked to find that i had dropped 13% in fluency and also had so many pods ungilded when I have been consistently practising strengthening skills nearly every day, since completing the course about two months ago. it was a huge disappointment and had me wondering what had happened. I had no idea that you carry out these tests and don't like being part of a test that I haven't been informed of or asked permission to be in. I've been a student of Duolingo for years and have highly recommended it to many people, but I am now considering looking for a different online language course, unless there is an apology from Duolingo and a promise to always request permission from your students if they want to be included in Duolingo testing.


      I don't think there needs to be an apology, they reversed the implementation. I also don't mind being part of a test group if it enhances the ultimate goal of learning. You probably agreed to this somewhere along the line in the fine print. But if DL is going to create A/B test groups, they should either leave the groups alone and change only new users or implement changes very slowly over time. I could handle doing even twice as many non-gold pods as I usually have, just not 50+. I am super appreciative of this free-product. Rosetta costs hundreds of dollars, is much more time consuming, and a beast on mobile devices. I feel overwhelmed by it. There is nothing even close to DUO in terms of leaning a language in a fun way, so Kudos to the team for that. I will be a paying subscriber when the option comes available. Everyone makes mistakes, and this BIG one was quickly reversed. Thank DUO!


      Same here, I practice daily and keep the bar of all classes at the highest level, but the calculated score suddenly dropped.

      I definitely agree that "don't like being part of a test that I haven't been informed of or asked permission to be in". I don't mind being in a test, but I should be informed.

      Hope this won't happen in the future.

      DL is a good tool to learn a new language and it helps me learn a language on my own pace.


      It is de-motivating for me to have my percentage go from 56 to 50% - although that is probably more accurate.

      Thank you for the explanation though!


      The fluency shouldn't motivate you, Mine is at 1% even though I've been using this account for French for almost two months now.


      It shouldn't and yet it does. Take a look at this link:


      Keeping up the streak on duolingo is the same as the seinfeld secret. I work hard to maintain my streak because it's been shown that doing a little bit of anything consistently over the long term has tremendous benefits. You can also look up the slight edge principle.

      The fluency indicator is a barometer for me on my progress. Even though it's not necessary, as I know I am making progress, but psychologically it's good to have the confirmation and therefore I do find it motivating.


      Hi Spongebobthereal - I guess you r using the app - fluency doesnt really work on there!


      Considering that the skill strengths, fluency percentages and other such measurements don't actually measure anything meaningful, there is little point playing about with them.

      I am a native English speaker and reasonably fluent (around C1) in Italian yet my skill strengths and fluency percentage are no better there than for languages in which I am far less fluent (A2 - B2) .

      Perhaps finding a more accurate way to measure progress and fluency would be useful because, for me at least, they are irrelevant.


      Judging from the level of complaining in the forums about it being too hard to keep things gold, why can't I get my fluency percentage higher, etc, etc, not to mention the fracas of the past 24 hours, these things are clearly highly motivating (or demotivating) to people. The populus Duolinguensis is not made up of rigorously optimizing automatons, but rather mostly the kind of neophytes who, even if they know it doesn't measure anything on any actually cognizable scale (a rare enough criterion) still find the inane fluency percentage motivating. So, in short, yes, there's all the reason in the world to monkey around with these things. None of them reflect anything on any pre-defined, real-world scales and they're simple and low-cost to adjust, yet they bring about real changes in behavior; in short they're exactly the kind of levers any policymaker dreams off.


      Worrying though, the thought of people playing mind games with other people, some of whom are quite young and might believe these wretched things to the point of being put off languages for life, or worse.


      I have to suspect that far more wind up put off when they don't feel themselves to be making much progress against the towering, impenetrable mass that learning a new language appears to be at the outset. Unlike people's perhaps gloomy self-perceptions, that fluency gauge will reliably go up with time spent. Skills will duly get regilded.


      You may have a point there but it almost always hits a plateau and then they become disheartened. Difficult one really.


      Indeed. Rename it "[Language] Progress Meter"; let it actually get to 100, this upon having gotten virtually the tree's entire vocab stock into active memory; and I think many would find it infinitely less obnoxious (both those who know enough to immediately find it obnoxious and those wondering why it just won't go any higher).


      @jrkhal I agree, maybe their A/B test did show them it was "better" but it is their definition of better which concerns me. From recent events and so called "improvements" which are anything but it seems as if "better" equates to more accounts, not to learning outcomes.

      This concerns me greatly as an account is still an account even if it has been deactivated or belongs to a troll. There is no way to close an account permanently on Duo and that fact alone skews the figures.

      I would love to be proved wrong and that "better" really did mean "better" but I have grave doubts about that. You may have more information than me but from everything I see here these days, it does not seem to be the case.


      An excellent suggestion but despite numerous attempts by people to persuade them to change the name, they remain firmly wedded to the highly misleading "Fluency meter". Heaven only knows why.



      which concerns me.

      Yep, me too. Hence my quote marks and the emoji. ;)


      @jrkhal Hahaha! and here I was thinking you had some insider information. :-))


      they remain firmly wedded to the highly misleading "Fluency meter". Heaven only knows why.

      Maybe A/B tests showed them that this name was the "better"... ? ;)


      It is probably advisable to read the very recent announcement here:


      Don't think I understand the connection?


      I agree. Motivation is nice, but accurate measures of knowledge are more motivating, since we know that the underlying algorithms behind all the measures we've been given aren't particularly relevant. It's frustrating that none of the measures introduced so far seem to accurately capture skill level, even within the A1-A2 range.

      If the A/B testing were actually going to help with that, I'd be all for it. But I question what "better" or "worse" outcome means in this case. Does it simply mean amount of time spent on the site, or does it include error rate? (And if so, does it distinguish between minor and major errors? I assume not.)

      I wish Duo staff would share whatever measures they're using to determine better or worse outcomes with users directly, instead of trying to write and roll back algorithms to produce shiny icons and wizard-behind-the-curtain numbers. Feedback in learning is only useful if the learner knows the feedback is accurate.


      I think better or worse probably just means people joining the site. Numbers of accounts seem to be the be all and end all of everything, even if said accounts are sock puppets or, alternatively, genuine accounts whose owners learn little or nothing because the metrics give them a flawed idea of their proficiency.

      Note, it can never take into account people closing their accounts permanently because Duo won't let you do tha. Instead, an account becomes deactivated but is nonetheless a Duo account, so if they are counted in the user statistics, those numbers are equally meaningless.


      Given the ads, I think pageviews are probably also counted. Most sites use that as a key metric when talking to advertisers. But yes, new accounts are probably a major component.

      I'd like, ideally, to believe that there are other metrics in use. But without anyone actually saying what they are, that's just me taking on faith that developers generally want to make a good product, even if the top-down word is that decisions must be based on userbase size. I mean, these folks were hired to make a language site. Even if the learning model in use doesn't seem to be based on much empirical research other than their own A/B testing, it seems like they ought to have some idea of what learning progress is and what they're trying to measure, even if that idea isn't realistic. Without knowing what it is, though, users have no opportunity to judge its usefulness for themselves.

      So yeah, if the numbers are made up and don't matter, what is everyone even doing? I really hope that's not the case at every level.


      When sentences appear that don't seem like the appertain to the unit in which they're been presented, that's them measuring learning outcomes ;)

      They've actually been somewhat forthcoming about all of this. Here's a couple of things to read:


      Thanks! I'd already read the first one, and it focuses solely on spacing for memorization. What it doesn't do is talk about language learning outside of word memory. (I've used Pimsleur's method and found that some parts of it work better than duo - for example, training pronunciation from the end of long words is better for speaking practice, but the TTS used here doesn't allow for it, I guess.)

      If all duo measures is word memory, then I agree with the suggestion you made earlier about changing "Fluency" to "Language Progress Meter" and being honest that it's just about word strength. Word memory is only one component of language knowledge, and duo just doesn't appear to be set up to measure some of the others. In that case, being clear about that is the best policy.


      I completely agree with you but I doubt we will ever know the answer. :-)


      Yes! It slays me how invested people are in their "fluency" percentage - it's such a random and arbitrary number. And even if it were wholly objective and quantitative ... what would even mean to be, say "50% fluent" in a language? Which 50%? I can speak but not write? I can speak and write half of it? I can get halfway through a conversation, then have a brain freeze?

      Presumably one is here to learn a language, and so one should concentrate on that - and not arbitrary game-like awards and rewards.


      I love the picture you conjure up Sierrajeff but you have missed another possibility. A complete conversation but with only half of each word spoken. :-))


      There has also been a recent notice about the current loss of activity steam.
      Please check here for more information about this issue.


      I’m glad that Duo staff has given us an opportunity to gradually improve our skills. Now I can take my time and slowly improve my skills before the change is fully implemented.


      I'm not sure I understand a word of what you said, but I love the fact that you are constantly striving for self improvement. I am, too, which is why I will continue joyfully with DuoLingo, regardless of my reduced fluency level. After all...what's in a number?? :-) Thank you for the explanation.


      I stopped taking the skills strengths seriously at all, as I use Duolingo to practice my skills in languages I already master because I learned them before (e.g. in high school). So I now just practice the skills using the test out option, but only once the theme has faded completely, and ignore the strengths feedback because it's no feedback, I just invent my own feedback. I still like the idea of the tree and the fact you can practice words in themes, especially the test out option makes that very efficient. Also the percentages I ignore. The feedback is just unreliable.


      Hey, I'm happy to have been included in the experiment, though I would be even happier if you had let me know immediately why my fluency score dropped suddenly.


      I have a problem that might be connected with this...I don't know... Some of my hungarian vocabulary needs to be improved (achievement bars are 4/5). OK, so I do the exercise to get to 5/5 again. Once. Nothing happens - the achievement bars are still at 4/5. Second time - the same story. This is really demotivating, because I want to improve my vocab but I can't - those stupid achievement bars just won't move up. Does anyone have a similar problem? What should I do? Köszönöm :)


      I’ve had this problem with Greek. I assume you’re doing Duolingo on the web, right? Try doing it on the app on your phone. I’ve found that some skills that just won’t turn gold on the web can turn gold when I do them on my phone.

      If you’re already trying on the app, try on the web.

      P.S. I assume you tried to guild the skills separately and not just used general practice. I’ve found that general practice hardly ever guilds skills on the app.


      I'll recommend the same thing as @FilipFilip17 Try strengthening from your phone. See my other discussion below:



      I hope I'm not to late to have a question (or two) answered here by someone more in the know.

      1) How long was the A/B testing in place? I had some incredibly far down skills decay and had probably done those skills close to a year ago.

      2) If the A/B test hasn't been going for a year, has the algorithm in general been reworked? It's both much harder to re-gold skills and they don't seem to stay that way for long.

      Side note: It seems important to share a little love for Duolingo at the moment. Despite it being a bit disheartening recently with decaying skills, this is really a wonderful resource, and I personally appreciate all of the time and effort that has made it possible for me to pursue German outside of a classroom.


      Hi JRollinson, I was wondering if the "test out of" feature was coming back anytime soon? It was really nice to use when I already knew a certain section of the content, but now I can't seem to find it... Is it gone or can I just not find it?


      that makes more sense


      why do you guys have to do this. what if i don't want to be part of a a/b test. i would like to be exempt from the a/b tests and everything returned back to normal for my account. if you want to do an a/b test make shure the people in it want to be part of the test.


      what if i don't want to be part of a a/b test.

      if you don't want to be part of any A/B test, then you'll have to stop using Duolingo but also Google and many many many other websites your (probably) using daily and that all run continuously such tests.


      i think they should have volunteers who would want to do the test be part of it not the ones who dont want to be part of a/b tests


      No, it would bias the result.
      It's exactly the basic principle of A/B test: being done on all users (any user is in one of the two groups of the A/B test) without telling anyone which group they belong to. Like that you can retrieve statistically relevant information.


      Thanks for the explanation. I had a feeling it was something like this. But it did help me a bit. I did apparently forget some words, even if they were gold. But it was a pain to strengthen almost my entire tree to find this. But still, in the long run, thanks for the explanation.


      I cry bull! As things seem to have been returned to their previous levels over night!


      Thank you, I'm very happy you're taking time to work it out and improve our learning capabilities


      I 1000000000% HATE you did this it is not a true fluency rating and seeing a 9% drop over night during a 135 day or so run is completly frustrating and demoralizing. I know I understand and habla español better than 47%. I can't emphasize enough how much I hate this, just remove fluency altogether or fix it.


      Hmmm. You want them to fix the fluency measure. What if that was difficult to do, and they weren't sure which way of measuring it worked best? In that situation, they'd do what any other website does, and do an A/B test.

      They've just spent a lot of time and effort doing an investigation precisely to improve the fluency measure. I'm sure the job isn't finished, and there's more to do, but they're doing what you want them to.

      The only thing they got wrong was announcing it 24 hours after ending the test, rather than 24 hours before.


      You mean hablo español? Perhaps that 9% drop is warranted? Sorry, couldn't resist :-)


      There is a user script on the wiki where you can hide the fluency (I have not tested it) :-)


      Have for anybody else fluency percentages suddenly increased? For two of my courses they are around 70% now and I think they were around 50% before.


      Have for anybody else fluency percentages suddenly increased?

      According to what several users mentioned in this discussion: yes.
      See for example, GaryLaRowe's comment or Stevestewart0001's comment.


      Thanks. I already read some of this thread yesterday but the percentages did not change till today for me. And they have never been this high.


      Nice (and rounded) streak! ;)


      Thanks for noticing. It is so pretty I'll postpone my practice for today a bit just to look at it longer ;)


      Two days ago my fluency strength dropped from 54% to 48%. I felt I had worked very hard to reach 54%. Since then, I have felt so discouraged that I have not been back on Duolingo and am looking for another way to learn Spanish.


      Thanks that is fine...My daughter in Canada will be pleased too.She is learning as well and couldn't understand why I was far ahead of her!!!


      Your methods are failing I like others are so very upset we look for where to tell the creators that we want to quit after a 10% loss in fluency rating, many of us have been studying more than 8 months and some more than a year. To be less than 50% fluent feels like a complete failure. I suggest fix it or lose more people including schools who use Duolingo.


      Leaving this particular hiccup aside, DuoLingo could do two things with Fluency that would make me very happy:

      1) Show me why my fluency has increased or decreased. For decreases in particular, this would lead me, in a constructive way, to improve weak skills; and,

      2) Give me the ability to hide/unhide Fluency altogether. I give it more weight than it is due, and it is really demotivating to me when it goes down.


      1: It's always because of word strength. If you want to improve your fluency score, strengthen any skills that aren't gold any more, and then use strengthen skills.

      2: There's a userscript that will turn it off for you. It's available here: http://duolingo.wikia.com/wiki/Duolingo_Userscripts


      Same for me, GeorgeHorv1. My daily pattern is: strengthen any badges that aren't gold; do additional five strengthening lessons; do four or five new lessons. Fluency hasn't budged.

      How does one go about getting DuoLingo, rather than just the community, to comment on what specifically is going on here?


      Well, this discussion was started by a duolingo member.
      Alternatively they probably have a contact email you can write to.


      Great info hughcparker! Thanks. Here - have a lingot.


      It looks like the indicator, at least for me, is not working. Since I have been demoted from 41% to 34% I have kept the tree gold by completing at least 5 lessons most days. I did 360xp this week alone and 2613 this month. I also completed about 4 new skills since and moved up a level to level 17. Despite all that my fluency hasn't budged from 34%.


      Are you on the app? The indicator doesn't recognise work done on the app (most platforms), but switch to the website version and you should see it rise again!


      Yes, I use the droid app. I did a few lessons on the web early on, about 10, but find the app much easier to use and have been using it exclusively since. I have seen my fluency number go up as I was using the app so I have to disagree that duo does not recognize work done on the app.


      Can I understand what you write about when I finished all lessions? :-D


      Would this have anything to do with why my fluency level went from like 56% to 29%? That's a huuuugee drop


      I have lost over 150 lingots presumably caused by the above. (I don't use them anyway!) Also my fluency has gone downfrom about 57% to much, much lower. I am going to have go back over all my topics which are showing only one or two bars - previously practically all were full! Is there going to be a fix for this? I love Duolingo but feel we have been "done"! I know Duolingo is free, but this is rather upsetting and confusing.


      I am happy that Duolingo is continually trying to improve their program, and I certainly understand that conducting such tests are an important port of that improvement process. Given that this is a free program, nor can I argue with Duolingo's right to conduct them (openly or quietly). I would make two suggestions though.

      <h1>1 List what tests are currently being conducted in any given curriculum, without directly informing participants which group they may happen to be in. Perhaps such a list already exists and I am unaware of it?</h1> <h1>2 I would also suggest that paid subscribers should have the right to opt out of being part of any group that would not to considered to be "the standard" or "Duolingo state of the art", if they choose to.</h1>


      I have absolutely no idea what you're talking about!!! This system is completely impossible to keep up with, you miss one day and LOADS of your strengths are demoted. WHY??? it's really beginning to piss me off.


      Says boosted-word-score students are doing worse than non-boosted-score students.

      Ignores all the complaints and discouragement from the non-boosted-score students.

      Methinks your data is biased.


      I appreciate the attempt to improve learning quality, but I don't think it is reasonable or necessary for me to have to redo lessons I have completely memorized daily. I would like to review the basics every now and then, as I already have a good understanding of them.


      Thank you. I now understand the odd fluctuations in my rating. I'm not a fan of the Fluency meter but considering how much I've learned from Duolingo I'm not going to complain about blind experiments. I appreciate continuing attempts to improve the site.


      why is my level always the same


      Because you didn't (yet) accumulated enough XP to reach the number of XPs corresponding to the next "level [of work put in the course]":

      The Support Pages' article "What are levels?" will tell you more about that.


      I just updated to skill levels and wish I hadn't. I just spent 30 minutes doing exercises that were ALL variations of "we are men, you are women". 3 steps full of exercises on just these words. Such a waste of time.


      It is not professional to share Duolingo score on Linkedin profile.

      Learn a language in just 5 minutes a day. For free.