Changing How We Display Learner Numbers
Starting today we’re making a change to the way we count and display the total number of learners in each of our courses. From now on we’ll be showing only active learner numbers in the Incubator, in our course listings, and on each individual course page.
Before, we counted and displayed all registered users in each course. Now we will only be showing active learners in each course in the last year—a much more accurate measure of current usage. Since we are no longer showing a cumulative number, this may appear like a significant drop in learners for a particular course, when in reality it is simply a change in how we display total learners. This gives everyone a better picture of current course information, instead of heavily favoring older courses that have been around longer. We are expecting this to help us and all of our wonderful contributors better understand course usage in order to make more informed decisions for improving your learning experience.
This is a big step in a sensible direction!
Do you have plans to break down the display further, perhaps numbers for active in last year, in last six months, last quarter, and last month?
I think a lot of us are data geeks and would love to see this information :)
EDIT to add: omigosh! Even with a "drop" and only looking at active users in the last year, those numbers are HUGE. Much larger than I anticipated. O.O
Oh, that's cool they're showing courses from all base languages now. Maybe they'll add a way to actually delete courses when they're the only one from a given base language. It looks like there are some kinks to work out yet. I think it's kind of cool and atmospheric that the learner counts are all displayed in the format for the base language of the tree, like "31.1L" for Hindi, but I suspect that might not be intended.
I treed out on Spanish at quite a low level, left it, and then found that loads more levels have been added and I'm not showing it as having completed it now. Which is kind of frustrating but then at least it gives me more to work on! Duo does seem to keep updating the courses, even the well established ones.
Hello Duolingo team,
Q1: What is the detail criteria for an "ACTIVE learner"?
- What reached minimum level
- Weekly and total XPs
- Breaks of 2-4 weeks or more (some users may come back e.g ~3-6 months later)
we counted and displayed all registered users in each course.
Ok, I am registered with French and Spanish on level 1 (0XP).
Q2: If I pass the first 2-4 skills and reached level 2 or 3, will I then be counted as an ACTIVE user, forever?
Q3: What does this mean for Czech, where I passed the first Intro skill (checking quickly audio with Camilo's "Duolingo Tree Enhancer" user script), got 30 total XP but where I am still on level 1?
Yes, I feel very sorry that I could not go on in 2017 when the new Czech course was hyped in forums (I think I wrote a draft but never posted that)....but I really needed to stick to Portuguese to keep making progress...
Es tut mir wirklich leid!
Q4: If users pass the first levels, but then stop (at a lower level), is the user then just registered, as he is not ACTIVE anymore, or still being counted in?
More concrete Portuguese examples:
I can see it for my Portuguese (English and German) clubs.
MANY learners initially joined with lower levels (not all from scratch), firstly made good progress with lots of XP for the first weeks (300XP++), but the drop out rate is usually before levels 10/11.
Some users even do not make it so far!
As I followed these users (web dashboard) I can say:
NO, these users did not (all) pickup the course again, many stayed inactive, did not progress further from the previous reached (old) level.
Some more concrete SHOF Portuguese learner from ANY languages statistics:
- 2637 Students (counted with 100+ day streak or alumni (or 100,000XP?))
- 3043 Trees (at least level 1+)
- 2086 @ Level 10+ = 68.55%
Q5: If users STOPPED at levels 4-9, will they keep being included in your statistic, or when do INACTIVE users fall out from your counting numbers?
Q6: What do you do with the remaining 31,45% according to the above statistic example?
Count them as active, inactive and/or when does it switch sides?
Thanks for taking your time.
If something is unclear please ask!
I am not an English native speaker; maybe I can (should?) improve something in my text for my posted comment?!
Best regards / Viele Grüße
Either I didn't understand the main post or you didn't . . . from my understanding, an "active" user is one who has completed at least one lesson in that language in the last year. So people who stopped using Duolingo two years ago will not be counted in any of the courses, and someone who has done, say, Spanish in the last year will be counted as an active user of that, but if he has neglected his French, then he will not be counted in the numbers of the course participants for French.
Actually this seems to be my problem.
I did not understand really much :-(
...that you aren't a native English speaker.
How long have you been studying English (sorry this is off-topic from the main thread).
More than 23 years (incl. education, chatting, IT/EDV business):
"(Late) Celebration of 600 days streak, finished Portuguese tree two times, personal background about me": https://forum.duolingo.com/comment/28094594
Quote HelpfulDuo: Now we will only be showing ACTIVE learners in each course in the last year —a much more accurate measure of current usage.
Thank you svrsheque for explicitely pointing me to the highlighted phrase about "last year" :-)
I have to admit:
ALL my questions have been initially phrased, not with this particular in mind!
I somehow read over the phrase, recognized it, did not understand it and must have put it off my mind.
I still do not understand well how it is exactly meant by staff :(
Q: Can a German native speaker maybe help?
With regard to the 3+ posts from Lrtward, 2GS, Gamekkeut, Stardust and others
about additional "last month(s)" to "last 6+ months" (of this year) detail statistics or general statistics about the 2-5 years before last year (<2017):
What does this help an established Duolingoianer or NEW user to make a decision based on the fact to only know about the statistics from last year?
How do those "ACTIVE learner" numbers reflect users who rolled into a course in the last seven months of 2018 and are quite active learning it?
Why don't you also include at least these people from 2018 and those people from the years before who managed to finish their tree, get their golden owl or reached a specific threshold language level?
However, personally I would probably not call these users then "ACTIVE learners" anymore if they stopped reviewing or did not get back to the course because of the crown level update in 2018.
They are really only shown (included) now when they were continually reviewing last year (2017)?
How many XPs per week / months?
If you only want to focus on the last year:
Why is it a "good thing" that you drop the user from those Incubator statistics who put 1-1,5 years or more of hard work into doing their course but had to move away from Duolingo because they either achieved their golden owl or reached level 25 before 01/01/2017 and did not comeback in 2017 or 2018?
I feel like the Incubator statistic should not only show "active learners" or only "enrolled users" but instead show a combination of all the above with preferably transparent defined thresholds?!?
Personally I want to make decisions in the future for enrolling into "old/new (including or exluding A/B) quality courses" myself based on these facts:
- How many "tips and notes" are actually available (e.g old EN-SP 13, new EN-SP 9)
- how many total users had enrolled
- to estimate how active might be the sentence discussions
- how many alternative words / sentences have already been accepted
- how huge the "reported/suggested sentence" queue still is to be cleared by contributors
Sorry, I can not really make these decisions only about "registered learners" (no filter) or "active learners" (fixed statistic from last year with no updates in 2018).
How do you even plan to show numbers for newly rolled out Incubator courses or finished beta testing in 2018, when the enrolled and active users from 2018 are not included?
A lot of those questions I can't answer, but it seems to me you might not understand what HelpfulDuo meant by "active user."
From my understanding (which may be incorrect), anyone who has gained any amount of XP in a certain course during the previous year will be counted as an active member of that course. This includes someone who gained 10 XP eleven months ago; this also includes someone who joined yesterday and gained some XP today. To be counted active, one does not have to have completed a lesson a year ago; one only has to have completed a lesson from a year ago to the present day. Also, I don't think this begins January 1, 2017; I think it's on a 365 day basis, not a calendar basis.
My apologies if you already knew this; I'm just trying to help.
The answer to all these is very simple. There is no definition of what is an active user. So as it is an arbitrary index, that was put for commercial reason (ads, how many, it is important for marketing). About learners of the less spoken languages it is important too, as language is a part of the local culture. So for instance English and Spanish have no need to show it, they are global actually.
There is no definition of what is an active user.
HelpfulDuo seemed (to me) to clearly define what they are counting as active users now, which I have already stated in a couple of posts here, one of which you just replied to, so I don't think I need to say it again. However, I'm curious as to why you think "active user" was not defined when it seems to me that it was?
So, if I've actively added the Spanish course to all seven of my accounts (solely for the trick to access the "words" page of other courses) —and I added them all within the last year— the fact I've earned 0 XP in them means that none of these will count as me being an "active user" for that course? 100% definitely?
Great!! I've always disliked the thought that I might be boosting its learner numbers. The Spanish course already being in the lead doesn't need this unfair extra advantage. ^^
Although, I can't personally see where HelpfulDuo defined "active" as specifically meaning "earned more than 0 XP". So I don't feel too confident just yet about believing this to be fact.
I do believe this is correct, yes. I'm not 100% sure . . . maybe more like 95%.
I'm not even sure that someone who has earned no XP was even counted in the old system, as it doesn't show up when you comment on Discussion pages, but I don't know about that.
Well, it seems to me that "active in the last year" means that you've actually done something in the course in the last year, and if you've not earned XP ever, then you definitely wouldn't have in the last year.
I'm not even sure that someone who has earned no XP was even counted in the old system, as it doesn't show up when you comment on Discussion pages, but I don't know about that.
It doesn't show up even if that person has earned 59 XP though, since it's only upon reaching level 2 in the course (60 XP) that it displays the course flag next to the username.
I think we were assuming someone only needs to earn 1 XP in a course within the last 365 days to be deemed "active"?
If a person does just the first lesson in a course (10 XP) and then doesn't use the site for a year, they are deemed "active" in that course for the whole year. Then, a year later; if that person does Timed Practice and answers just a single question (1 XP); then they'll be at 11 XP, the flag still won't show next to their username as they'll still be level 1, and yet they'll be regarded as "active" in that course for a second year in a row... Can just keep earning 1 XP on each anniversary. :D
may I ask why you need seven accounts?
Short answer: Risk avoidance, mobile app quarantine, A/B test selection, and various testing purposes.
I joined on Duolingo only to learn Japanese, but there was no Japanese course. So I started the reverse tree that was currently in beta instead.
When the actual Japanse course finally came into being, its alpha test was restricted to iOS only. I don't use the mobile app, and my mobile devices don't have iOS anyway. I wanted to access the alpha on web via a trick Horako224 mentioned in the discussions. However, from what jrikhal said in that thread, it sounded potentially risky to do it on my main account. So I made a secondary account ("testmoogle") and started the Japanese course (and Korean) on web long before they were officially available on web.
When the beta version of the Japanese course released only on iOS and Android (it didn't release on web until several months after this), at that point I wanted to compare the unreleased web version I was using to the officially released Android version. Web was typing full sentences only, whereas the mobile app was just word bank & match the tiles exercises. I didn't want my testmoogle account to be tainted by this feeble version of the course. So I created a couple of "droidmoogle" accounts to check out the mobile app version of the course.
Later, I ended up making some additional testmoogle accounts to test various other things (the word page trick, comparing A/B test differences, testing various userscripts I make, etc.)
Thus, I ended up with seven accounts. However, I only use two daily: the account I'm on now and my main account. ^^
Yep, but it does give a clearer picture of activity on Duolingo as a whole when around 10000 members have at least a 100 day streak, and then we have the total count of possibly active members in the past year numbering in the millions. We aren’t sure of what Duolingo considers active, either. Were these users active once a week for a certain time, every month... every six months.. at least once in the past year. I’d love to see more details.
These aren't statistics, these are marketing numbers.
Statistics deals with open data sets that can be independently analysed and examined by any interested person. Duolingo's numbers are very far from open. We're not even given precise definitions of what these numbers measure, a sure sign of marketing vs honest reporting.
Duolingo has a long history of self-promotional spin in this area.
It's a bit unfortunate that you aren't displaying active learners for ALL courses on the "All Languages" page. There are only two that aren't from languages with English as the base language. But the numbers can still be accessed by clicking on the individual language - for example, you can see the people learning English from Arabic by clicking on "Arabic." Is there a reason for this change?
This adjustment makes a lot of sense but I would like more information on what constitutes an active learner. I mix and match a lot of different languages but I consider myself active with all of them as long as I'm on DL every day. Not sure if that counts in the overall numbers though.
There are a lot of upvotes here, but I am not sure why. I am guessing it is because Duo users are invested in the site and appreciate being informed. But, while this news is relevant to monetization on the business end, not sure how it impacts individual language learners. We clearly appreciate Duolingo, which is why we are here. Can you elaborate on your final statement on how this should be improving our learning experience?
Thank you for the update, I have a note for this (you can downvote it if you are against me)
Before, we can count the total number of learners, but by this update, our announces will be different and weird in the same time in the incubator, for instance: Every one, two, three, five, ten ... millions learners, we have an announce for that.
And now, what is the benefit of this confusing update?, it does not mean anything frankly, for example: English course for Arabic speakers has 15 millions learners at this moment, when the team reached to 10 millions, they have been written an announce for that, and by this update we have 4 millions learners as an active???
Team work must focus in another thing from the best, anyway, thank you so much for everything :)
That's an excellent decision, and much more meaningful than counting everyone who's ever signed up for a language. That change might also explain why it's been so long since the learner numbers were last updated (I think it's been a month at least). Hope the updates will appear more regularly again! Thanks!
Why does "I speak all languages" now no longer show all the courses? Is this a new bug or an intentional change?
For example: It only shows one course for learning English. (This happens to be the one for Spanish speakers.)
It now seems to be only showing one base language for each target language. On my other account where my site language is set to Japanese, this doesn't even list the course I'm currently doing! It shows the Spanish speakers' version instead of the Japanese speakers' one.
* Update * (2018-07-27)
Thankfully this is now completely fixed. Nice one Duo! ^^
That is such unlucky bad timing for us on the Welsh course. Our cumulative numbers recently passed 1 million. This is a huge symbolic number since it's the notional target set by the Welsh government to be achieved by 2050 (from around 600k at the moment - nothing if not unambitious is the Welsh government). In 12 days we are holding a cememony to celebrate this at the premier Welsh cultural event of the year, the National Eisteddfod. We are hoping that no-one will notice the change before our event :-)
I think the word you´re looking for is "competence."
It would be nice to know the rate of finished trees, although I suspect Duolingo is unlikely to release it. (It is rumored to be very low. The number of people who dabble for a few lessons and then give up is probably quite high.) Also, the numbers are not directly comparable, since different trees are of different lengths and degrees of difficulty. For that matter, numbers for the same language at different times may not be comparable, as the trees grow in length and, at least occasionally, in degree of difficulty.
If you want the info on "finished trees", you can find it, if by "finished trees" you mean golden owls. Just go to Duome.eu and look in the right sidebar, and you'll find a list of the golden owls for all languages (at least those taught from English). You may have to wait a moment for it to load (the left-hand column, which is very long, will load first), but it will come up right after that.
Salivanto/Tomaso, well, yes, you'd think there'd be more, and maybe there are. I'm wondering what happens to the count when people temporarily lose their golden owls when the language goes to 2.0. This just happened in Esperanto. So I lost my golden owl, though within a few weeks, I'll get it back!
But even if we just take the numbers at face value, Esperanto is doing pretty darn good! I just did a little math (dividing learner numbers by golden owl numbers) to find out how many active learners there are for each golden owl. And of the 14 languages I checked, Esperanto comes out on top!
That is, one of every 1,066 learners has earned a golden owl, while it took 9,039 Spanish learners for each golden owl earned!
Here are the learner numbers per golden owl in:
Esperanto: 1,066 Swedish: 1,486 Portuguese: 2,047 Italian: 2,185 Dutch: 3,437 French: 3,948 Russian: 4,228 German: 4,230 Turkish: 4,360 Greek: 5,041 Spanish: 9,039.8 Chinese: 10,867 Japanese: 11,025 Korean: 28,602
One of the factors for low golden owl levels is probably that the languages have been added fairly recently, as in the case of Chinese, Korean, and also Japanese.
I'm just thinking in terms of there being at least 377 fluent Esperanto speakers who did the course just to see what it was like. Maybe I'm unusual in that regard.
One finisher out of 1000 starters seems pretty good - but when you consider that over the lifetime of the course, 1.3 million signed up and 377 are showing in duome, That's one finisher in almost 3500 starters. I guess that's not totally out of scale - but I've always assumed that lots of curious fluent speakers where doing the course too.
The relative numbers probably just mean that more people wish they spoke Spanish than wish they spoke Esperanto. :-)
The explanation at the top here will probably clear the matter up:
All listings on duome are for those listed on duome, which is far from everyone. Notably, it requires having been looked up on duome (by oneself or others) and then meeting the criteria for listing, which aren't 100% clear, but by and large I think it's 100-day streak or 100,000 XP. As you can see, the 300-some tree finishers are out of a denominator of less than 1600 listed learners of Esperanto.
Thanks. 300 really did seem impossibly low. 300 out of 1600 seems plausible for a very active subset of the "1.7 million" who ever tried one of the Eo courses.
I guess the answer is "nobody knows." I would not expect there to be 300,000 finishers of the Eo course. I've heard number between 30 and 100 per day finish the course. I think the 100 per day number was one person's extrapolation and it's now being passed around as a statistic.
I hadn't thought of that! I'm sure a lot of Duo students don't even know about duome.eu. I didn't find out about it until after the switch to the crowns system, and only because I read about it in one of the discussions. So that (and the 100-day streak requirement) may well account for a much lower number...
Why have you changed the format of the lessons I now find that I am going to have to complete thousands of lessons from scratch right through the course. I don't think that it gives much encouragement in fact to me it has had the opposite effect in as much as I really don't feel like continuing the course at the moment. I will go away gather my thoughts and then see what happens. I am totally disappointed with the new layout.
One year is a very broad definition of active: you still want the numbers up incredibly high. Why not just show us how many logged on for the week? Be lazy about it and just update the numbers every Monday morning. And why not put the exact number, for example, 3,458,523. Computers compute and count, and we humans can actually read and understand such numbers. Then you really are telling us the size of our language learning communities. Your one-year numbers are still way off and in a sense keeping the true activeness of your website a secret.