Changing How We Display Learner Numbers
Starting today we’re making a change to the way we count and display the total number of learners in each of our courses. From now on we’ll be showing only active learner numbers in the Incubator, in our course listings, and on each individual course page.
Before, we counted and displayed all registered users in each course. Now we will only be showing active learners in each course in the last year—a much more accurate measure of current usage. Since we are no longer showing a cumulative number, this may appear like a significant drop in learners for a particular course, when in reality it is simply a change in how we display total learners. This gives everyone a better picture of current course information, instead of heavily favoring older courses that have been around longer. We are expecting this to help us and all of our wonderful contributors better understand course usage in order to make more informed decisions for improving your learning experience.
This is a big step in a sensible direction!
Do you have plans to break down the display further, perhaps numbers for active in last year, in last six months, last quarter, and last month?
I think a lot of us are data geeks and would love to see this information :)
EDIT to add: omigosh! Even with a "drop" and only looking at active users in the last year, those numbers are HUGE. Much larger than I anticipated. O.O
I can only agree with you. The numbers dropped WAY way more than I thought it would.
They did. Still, it's nice to see something more reflective of reality and less reflective of advertising, more "people in seats" rather than "number of tickets sold."
What about people who already completed the entire course and don’t use it anymore?
I think they're still advertising "number of tickets sold" but just limiting it to "tickets sold in the last year."
Well, not exactly . . . if you bought your ticket three years ago but still are sitting in your seat (participating in the course) then you're counted.
Close enough. You said elsewhere that Helpfulduo did indeed define "Active". I must be thick because I don't see it. Could you point it out for me.
For some reason it shows me as studying English for speakers of Korean, which I don't think I've even glanced at. But I think the new numbers are a step in the right direction!
Oh, that's cool they're showing courses from all base languages now. Maybe they'll add a way to actually delete courses when they're the only one from a given base language. It looks like there are some kinks to work out yet. I think it's kind of cool and atmospheric that the learner counts are all displayed in the format for the base language of the tree, like "31.1L" for Hindi, but I suspect that might not be intended.
You are everywhere on this site. I strive for your diligence. One day...someday...
My question is if people who have finished the tree are still listed as learners since I feel they should be counted even though they may not be working on that particular tree anymore. Although that might get confusing with the endless number of upgrades.
I’m curious about this too. Since streaks are user-specific already, is that kept for learner data...
When I looked at the courses page, for some reason, Norwegian and Dutch were not ticked as "learning". This is confusing. They are not my only finished courses and I am sure I have worked on them recently. However, I may not have reached a new crown.
No, don't see any significant problems, just the lack of an insignificant marker.
Hopefully you both have not lost your course progress?
What happened when you clicked on the "switch" button?
Is everything right and back to normal state?
exactly the same situation here for those two languages for me. i do them every day (well, regularly at least 6 days a week).
That would be interesting as a separate statistic, to know how many have started, are active and have "finished" a course.
I treed out on Spanish at quite a low level, left it, and then found that loads more levels have been added and I'm not showing it as having completed it now. Which is kind of frustrating but then at least it gives me more to work on! Duo does seem to keep updating the courses, even the well established ones.
Hello Duolingo team,
Q1: What is the detail criteria for an "ACTIVE learner"?
- What reached minimum level
- Weekly and total XPs
- Breaks of 2-4 weeks or more (some users may come back e.g ~3-6 months later)
we counted and displayed all registered users in each course.
Ok, I am registered with French and Spanish on level 1 (0XP).
Q2: If I pass the first 2-4 skills and reached level 2 or 3, will I then be counted as an ACTIVE user, forever?
Q3: What does this mean for Czech, where I passed the first Intro skill (checking quickly audio with Camilo's "Duolingo Tree Enhancer" user script), got 30 total XP but where I am still on level 1?
Yes, I feel very sorry that I could not go on in 2017 when the new Czech course was hyped in forums (I think I wrote a draft but never posted that)....but I really needed to stick to Portuguese to keep making progress...
Es tut mir wirklich leid!
Q4: If users pass the first levels, but then stop (at a lower level), is the user then just registered, as he is not ACTIVE anymore, or still being counted in?
More concrete Portuguese examples:
I can see it for my Portuguese (English and German) clubs.
MANY learners initially joined with lower levels (not all from scratch), firstly made good progress with lots of XP for the first weeks (300XP++), but the drop out rate is usually before levels 10/11.
Some users even do not make it so far!
As I followed these users (web dashboard) I can say:
NO, these users did not (all) pickup the course again, many stayed inactive, did not progress further from the previous reached (old) level.
Some more concrete SHOF Portuguese learner from ANY languages statistics:
- 2637 Students (counted with 100+ day streak or alumni (or 100,000XP?))
- 3043 Trees (at least level 1+)
- 2086 @ Level 10+ = 68.55%
Q5: If users STOPPED at levels 4-9, will they keep being included in your statistic, or when do INACTIVE users fall out from your counting numbers?
Q6: What do you do with the remaining 31,45% according to the above statistic example?
Count them as active, inactive and/or when does it switch sides?
Thanks for taking your time.
If something is unclear please ask!
I am not an English native speaker; maybe I can (should?) improve something in my text for my posted comment?!
Best regards / Viele Grüße
Either I didn't understand the main post or you didn't . . . from my understanding, an "active" user is one who has completed at least one lesson in that language in the last year. So people who stopped using Duolingo two years ago will not be counted in any of the courses, and someone who has done, say, Spanish in the last year will be counted as an active user of that, but if he has neglected his French, then he will not be counted in the numbers of the course participants for French.
Actually this seems to be my problem.
I did not understand really much :-(
what, czech hyped in 2017? you may want to consider the following in adjusting your questions:
Now we will only be showing active learners in each course in the last year
Thomas, I see your posts all the time, and you just STUNNED me with divulging that you aren't a native English speaker. How long have you been studying English (sorry this is off-topic from the main thread).
...that you aren't a native English speaker.
How long have you been studying English (sorry this is off-topic from the main thread).
More than 23 years (incl. education, chatting, IT/EDV business):
"(Late) Celebration of 600 days streak, finished Portuguese tree two times, personal background about me": https://forum.duolingo.com/comment/28094594
Quote HelpfulDuo: Now we will only be showing ACTIVE learners in each course in the last year —a much more accurate measure of current usage.
Thank you svrsheque for explicitely pointing me to the highlighted phrase about "last year" :-)
I have to admit:
ALL my questions have been initially phrased, not with this particular in mind!
I somehow read over the phrase, recognized it, did not understand it and must have put it off my mind.
I still do not understand well how it is exactly meant by staff :(
Q: Can a German native speaker maybe help?
With regard to the 3+ posts from Lrtward, 2GS, Gamekkeut, Stardust and others
about additional "last month(s)" to "last 6+ months" (of this year) detail statistics or general statistics about the 2-5 years before last year (<2017):
What does this help an established Duolingoianer or NEW user to make a decision based on the fact to only know about the statistics from last year?
How do those "ACTIVE learner" numbers reflect users who rolled into a course in the last seven months of 2018 and are quite active learning it?
Why don't you also include at least these people from 2018 and those people from the years before who managed to finish their tree, get their golden owl or reached a specific threshold language level?
However, personally I would probably not call these users then "ACTIVE learners" anymore if they stopped reviewing or did not get back to the course because of the crown level update in 2018.
They are really only shown (included) now when they were continually reviewing last year (2017)?
How many XPs per week / months?
If you only want to focus on the last year:
Why is it a "good thing" that you drop the user from those Incubator statistics who put 1-1,5 years or more of hard work into doing their course but had to move away from Duolingo because they either achieved their golden owl or reached level 25 before 01/01/2017 and did not comeback in 2017 or 2018?
I feel like the Incubator statistic should not only show "active learners" or only "enrolled users" but instead show a combination of all the above with preferably transparent defined thresholds?!?
Personally I want to make decisions in the future for enrolling into "old/new (including or exluding A/B) quality courses" myself based on these facts:
- How many "tips and notes" are actually available (e.g old EN-SP 13, new EN-SP 9)
- how many total users had enrolled
- to estimate how active might be the sentence discussions
- how many alternative words / sentences have already been accepted
- how huge the "reported/suggested sentence" queue still is to be cleared by contributors
Sorry, I can not really make these decisions only about "registered learners" (no filter) or "active learners" (fixed statistic from last year with no updates in 2018).
How do you even plan to show numbers for newly rolled out Incubator courses or finished beta testing in 2018, when the enrolled and active users from 2018 are not included?
A lot of those questions I can't answer, but it seems to me you might not understand what HelpfulDuo meant by "active user."
From my understanding (which may be incorrect), anyone who has gained any amount of XP in a certain course during the previous year will be counted as an active member of that course. This includes someone who gained 10 XP eleven months ago; this also includes someone who joined yesterday and gained some XP today. To be counted active, one does not have to have completed a lesson a year ago; one only has to have completed a lesson from a year ago to the present day. Also, I don't think this begins January 1, 2017; I think it's on a 365 day basis, not a calendar basis.
My apologies if you already knew this; I'm just trying to help.
The answer to all these is very simple. There is no definition of what is an active user. So as it is an arbitrary index, that was put for commercial reason (ads, how many, it is important for marketing). About learners of the less spoken languages it is important too, as language is a part of the local culture. So for instance English and Spanish have no need to show it, they are global actually.
There is no definition of what is an active user.
HelpfulDuo seemed (to me) to clearly define what they are counting as active users now, which I have already stated in a couple of posts here, one of which you just replied to, so I don't think I need to say it again. However, I'm curious as to why you think "active user" was not defined when it seems to me that it was?
So, if I've actively added the Spanish course to all seven of my accounts (solely for the trick to access the "words" page of other courses) —and I added them all within the last year— the fact I've earned 0 XP in them means that none of these will count as me being an "active user" for that course? 100% definitely?
Great!! I've always disliked the thought that I might be boosting its learner numbers. The Spanish course already being in the lead doesn't need this unfair extra advantage. ^^
Although, I can't personally see where HelpfulDuo defined "active" as specifically meaning "earned more than 0 XP". So I don't feel too confident just yet about believing this to be fact.
I do believe this is correct, yes. I'm not 100% sure . . . maybe more like 95%.
I'm not even sure that someone who has earned no XP was even counted in the old system, as it doesn't show up when you comment on Discussion pages, but I don't know about that.
Well, it seems to me that "active in the last year" means that you've actually done something in the course in the last year, and if you've not earned XP ever, then you definitely wouldn't have in the last year.
I'm not even sure that someone who has earned no XP was even counted in the old system, as it doesn't show up when you comment on Discussion pages, but I don't know about that.
It doesn't show up even if that person has earned 59 XP though, since it's only upon reaching level 2 in the course (60 XP) that it displays the course flag next to the username.
I think we were assuming someone only needs to earn 1 XP in a course within the last 365 days to be deemed "active"?
If a person does just the first lesson in a course (10 XP) and then doesn't use the site for a year, they are deemed "active" in that course for the whole year. Then, a year later; if that person does Timed Practice and answers just a single question (1 XP); then they'll be at 11 XP, the flag still won't show next to their username as they'll still be level 1, and yet they'll be regarded as "active" in that course for a second year in a row... Can just keep earning 1 XP on each anniversary. :D
Ah yes, that's very true. I forgot about that. (As I said, I wasn't sure about it not showing up.)
So, I do believe your seven accounts won't show for the Spanish course participants now, if they ever did . . . may I ask why you need seven accounts?
may I ask why you need seven accounts?
Short answer: Risk avoidance, mobile app quarantine, A/B test selection, and various testing purposes.
I joined on Duolingo only to learn Japanese, but there was no Japanese course. So I started the reverse tree that was currently in beta instead.
When the actual Japanse course finally came into being, its alpha test was restricted to iOS only. I don't use the mobile app, and my mobile devices don't have iOS anyway. I wanted to access the alpha on web via a trick Horako224 mentioned in the discussions. However, from what jrikhal said in that thread, it sounded potentially risky to do it on my main account. So I made a secondary account ("testmoogle") and started the Japanese course (and Korean) on web long before they were officially available on web.
When the beta version of the Japanese course released only on iOS and Android (it didn't release on web until several months after this), at that point I wanted to compare the unreleased web version I was using to the officially released Android version. Web was typing full sentences only, whereas the mobile app was just word bank & match the tiles exercises. I didn't want my testmoogle account to be tainted by this feeble version of the course. So I created a couple of "droidmoogle" accounts to check out the mobile app version of the course.
Later, I ended up making some additional testmoogle accounts to test various other things (the word page trick, comparing A/B test differences, testing various userscripts I make, etc.)
Thus, I ended up with seven accounts. However, I only use two daily: the account I'm on now and my main account. ^^
Hm, interesting! That does make sense, I guess; I've only ever had one account, although I haven't done anything "risky." I didn't know that this account wasn't your main one! (Although maybe I should have been able to infer it from the "test" in your username.)
That unfortunately only includes those who've made it into the Halls of Fame.
Yep, but it does give a clearer picture of activity on Duolingo as a whole when around 10000 members have at least a 100 day streak, and then we have the total count of possibly active members in the past year numbering in the millions. We aren’t sure of what Duolingo considers active, either. Were these users active once a week for a certain time, every month... every six months.. at least once in the past year. I’d love to see more details.
Duolingo members aren't automatically included on Duome. Somebody has to have looked them up there, which creates huge selection bias as to who's there even among the population that meets one of the criteria for listing.
A little change but a good one, I think. The new numbers should much more accurately reflect site activity. Although, seeing all the numbers drop is a little bit sad - especially since Welsh had only just broken 1,000,000 users.
These aren't statistics, these are marketing numbers.
Statistics deals with open data sets that can be independently analysed and examined by any interested person. Duolingo's numbers are very far from open. We're not even given precise definitions of what these numbers measure, a sure sign of marketing vs honest reporting.
Duolingo has a long history of self-promotional spin in this area.
This is a sensible choice, showing active learners in certain periods of time, e.g 2.3 million active in a week, 7.9 active in a month, would also be good! The number display would be useful to see how many and how much effort people are putting into their language learning.
I’d like if it was like
X Active users this week
X active users this month
X active users past 6 months
It's a bit unfortunate that you aren't displaying active learners for ALL courses on the "All Languages" page. There are only two that aren't from languages with English as the base language. But the numbers can still be accessed by clicking on the individual language - for example, you can see the people learning English from Arabic by clicking on "Arabic." Is there a reason for this change?
The "All Languages" option was removed a few weeks ago, and replaced with what Multi0Lingual4 described sometime this week. Though there are three listed that don't have English as the base language.
That is true, but what I think Jessica4201 is saying is that the "All Languages" tab was removed for a little bit (I tried to use it but couldn't find it, so I can verify this), and now that it's replaced, it doesn't actually have all languages, like it is supposed to do.
Looks like it's fixed now! The ordering at the bottom is a bit strange. Esperanto for Portuguese is listed after Spanish for Italian, which is still in the Incubator for the moment.
This adjustment makes a lot of sense but I would like more information on what constitutes an active learner. I mix and match a lot of different languages but I consider myself active with all of them as long as I'm on DL every day. Not sure if that counts in the overall numbers though.
There are a lot of upvotes here, but I am not sure why. I am guessing it is because Duo users are invested in the site and appreciate being informed. But, while this news is relevant to monetization on the business end, not sure how it impacts individual language learners. We clearly appreciate Duolingo, which is why we are here. Can you elaborate on your final statement on how this should be improving our learning experience?
126 million English to Spanish now down to 20 million? Oof. Now where was the statistic of having 200 million users...
They still do, just not active. Which is yet to be defined. And give take multiple accounts, bots, etc etc.
Which is yet to be defined.
What do you mean? It seemed clearly defined enough for me.
That claim wasn't just for the Spanish from English course was it? There are 81 courses available from a couple dozen base languages.
Thank you for the update, I have a note for this (you can downvote it if you are against me)
Before, we can count the total number of learners, but by this update, our announces will be different and weird in the same time in the incubator, for instance: Every one, two, three, five, ten ... millions learners, we have an announce for that.
And now, what is the benefit of this confusing update?, it does not mean anything frankly, for example: English course for Arabic speakers has 15 millions learners at this moment, when the team reached to 10 millions, they have been written an announce for that, and by this update we have 4 millions learners as an active???
Team work must focus in another thing from the best, anyway, thank you so much for everything :)
This is a good change, it will be interesting to see the real numbers in each course.
So if I have a language on my profile, but I never use it anymore even though I'm adamantly learning a different language on Duolingo, would I still be considered an active learner in both languages?
It may be a bug but Dutch and Norwegian are not ticked as "learning" for me. I'm sure that I have worked on these languages recently but I may not have reached a new crown. So, maybe this makes me an inactive learner.
Describing someone who's used Duolingo once or twice nearly a year ago as active seems like a massive stretch. Why can't they be more realistic and measure monthly active users like Facebook, Instagram and Snapchat?
Good. I quoted user numbers in a post once and they seemed unrealistically high.
haha I was wondering how many of us were active learners. I knew Spanish was nowhere near 100 million.
It's really amazing to see a company be transparent and honest at their own "expense" (the perception, not reality, of smaller number). I don't think I can be more grateful for the site and it's users.
Please add Latin! There are many people wanting to learn this amazing language, including me!
Thanks for the information. Also thank you for doing a wonderful job here. I am learning a lot. Again , Thanks.
On the language page. Slide down your user name to see an "add language option" That will give you the stats; you don't have to add a language in the process.
That's an excellent decision, and much more meaningful than counting everyone who's ever signed up for a language. That change might also explain why it's been so long since the learner numbers were last updated (I think it's been a month at least). Hope the updates will appear more regularly again! Thanks!
Why does "I speak all languages" now no longer show all the courses? Is this a new bug or an intentional change?
For example: It only shows one course for learning English. (This happens to be the one for Spanish speakers.)
It now seems to be only showing one base language for each target language. On my other account where my site language is set to Japanese, this doesn't even list the course I'm currently doing! It shows the Spanish speakers' version instead of the Japanese speakers' one.
* Update * (2018-07-27)
Thankfully this is now completely fixed. Nice one Duo! ^^
With things that wacky, I assume it's a bug and feel myself justified in submitting bug reports. If there were documentation of intended functionality, I'd check it, but there's not.
That's good for the apps statistics and for reflecting more accurate picture of the app usage. :)
Yes! I've wondered about the "actual learners" for a while. What a great move :)
That is such unlucky bad timing for us on the Welsh course. Our cumulative numbers recently passed 1 million. This is a huge symbolic number since it's the notional target set by the Welsh government to be achieved by 2050 (from around 600k at the moment - nothing if not unambitious is the Welsh government). In 12 days we are holding a cememony to celebrate this at the premier Welsh cultural event of the year, the National Eisteddfod. We are hoping that no-one will notice the change before our event :-)
Is there any way to drop a course from your profile? I know I won't do any more lessons in Portuguese, for example, so I would like to drop it from my "languages" list.
THANK YOU!!! Guess I should have looked around a little more... but appreciate the assist!
I think this is a really good choice to make, but while I looked at the incubator I noticed something:It says that Indonesian is 100 percent done, but the estimated launch time is 17th of august;Why is this?
17th of August was Proclamation of Indonesian Independence. Maybe Duolingo wants to release the Indonesian course when we're celebrating our Independence Day.
Hi Ferid1453, yours is a good question but not related to the topic of this discussion. Please start a new discussion so that your question will get the attention it deserves, and to avoid "hijacking" this discussion onto another topic. Thank you!
"100 percent," as I understand it refers to the percentage of sentences completed. There is still some additional work that needs to be done, once that point has been reached, before the course can actually be released.
So will the new learner count be updated every year now?
I believe it will be updated every day. I think it's on a 365 day period, not just from the beginning of 2017.
What about the clubs? I think there are only a few people left active i our club.
Interesting, first time I saw the page. Great for inquiring minds who want to know. It cracked me up when I saw Klingon listed as a language. Then I cracked up more when I realized it's not a joke. What's sad is that Klingon has 10 time more active users than Hindi.
Also would be awesome to the data lovers, the rate of finished trees and a ranking of the learners in each tree like in memrise to incentive the competence.
I think the word you´re looking for is "competence."
It would be nice to know the rate of finished trees, although I suspect Duolingo is unlikely to release it. (It is rumored to be very low. The number of people who dabble for a few lessons and then give up is probably quite high.) Also, the numbers are not directly comparable, since different trees are of different lengths and degrees of difficulty. For that matter, numbers for the same language at different times may not be comparable, as the trees grow in length and, at least occasionally, in degree of difficulty.
If you want the info on "finished trees", you can find it, if by "finished trees" you mean golden owls. Just go to Duome.eu and look in the right sidebar, and you'll find a list of the golden owls for all languages (at least those taught from English). You may have to wait a moment for it to load (the left-hand column, which is very long, will load first), but it will come up right after that.
Those numbers don't seem right. Is it saying that only 377 people finished the Esperanto course? That seems impossibly low.
Salivanto/Tomaso, well, yes, you'd think there'd be more, and maybe there are. I'm wondering what happens to the count when people temporarily lose their golden owls when the language goes to 2.0. This just happened in Esperanto. So I lost my golden owl, though within a few weeks, I'll get it back!
But even if we just take the numbers at face value, Esperanto is doing pretty darn good! I just did a little math (dividing learner numbers by golden owl numbers) to find out how many active learners there are for each golden owl. And of the 14 languages I checked, Esperanto comes out on top!
That is, one of every 1,066 learners has earned a golden owl, while it took 9,039 Spanish learners for each golden owl earned!
Here are the learner numbers per golden owl in:
Esperanto: 1,066 Swedish: 1,486 Portuguese: 2,047 Italian: 2,185 Dutch: 3,437 French: 3,948 Russian: 4,228 German: 4,230 Turkish: 4,360 Greek: 5,041 Spanish: 9,039.8 Chinese: 10,867 Japanese: 11,025 Korean: 28,602
One of the factors for low golden owl levels is probably that the languages have been added fairly recently, as in the case of Chinese, Korean, and also Japanese.
I'm just thinking in terms of there being at least 377 fluent Esperanto speakers who did the course just to see what it was like. Maybe I'm unusual in that regard.
One finisher out of 1000 starters seems pretty good - but when you consider that over the lifetime of the course, 1.3 million signed up and 377 are showing in duome, That's one finisher in almost 3500 starters. I guess that's not totally out of scale - but I've always assumed that lots of curious fluent speakers where doing the course too.
The relative numbers probably just mean that more people wish they spoke Spanish than wish they spoke Esperanto. :-)
The explanation at the top here will probably clear the matter up:
All listings on duome are for those listed on duome, which is far from everyone. Notably, it requires having been looked up on duome (by oneself or others) and then meeting the criteria for listing, which aren't 100% clear, but by and large I think it's 100-day streak or 100,000 XP. As you can see, the 300-some tree finishers are out of a denominator of less than 1600 listed learners of Esperanto.
Thanks. 300 really did seem impossibly low. 300 out of 1600 seems plausible for a very active subset of the "1.7 million" who ever tried one of the Eo courses.
I guess the answer is "nobody knows." I would not expect there to be 300,000 finishers of the Eo course. I've heard number between 30 and 100 per day finish the course. I think the 100 per day number was one person's extrapolation and it's now being passed around as a statistic.
I hadn't thought of that! I'm sure a lot of Duo students don't even know about duome.eu. I didn't find out about it until after the switch to the crowns system, and only because I read about it in one of the discussions. So that (and the 100-day streak requirement) may well account for a much lower number...
Thank you for keeping us updated. The changes made and exercises added are very much appreciated.
Why have you changed the format of the lessons I now find that I am going to have to complete thousands of lessons from scratch right through the course. I don't think that it gives much encouragement in fact to me it has had the opposite effect in as much as I really don't feel like continuing the course at the moment. I will go away gather my thoughts and then see what happens. I am totally disappointed with the new layout.
Reminder: You do not, in fact, have to do anything that you do not want to do or that you do not feel is conducive to your learning.
So does that mean if we are inactive on a language we will lose our progress?
My thinking is if they're removing inactive users from their counting are they removing progress too?
I'm sure they can count users any way they want without removing their progress.
One year is a very broad definition of active: you still want the numbers up incredibly high. Why not just show us how many logged on for the week? Be lazy about it and just update the numbers every Monday morning. And why not put the exact number, for example, 3,458,523. Computers compute and count, and we humans can actually read and understand such numbers. Then you really are telling us the size of our language learning communities. Your one-year numbers are still way off and in a sense keeping the true activeness of your website a secret.