Οpen letter to Duolingo team and Dr. Luis
Dear Duolingo team and Dr. Luis von Ahn.
With all due respect, I write to express my frustration about the recent changes.Since Duolingo is free I can't complain as a customer,but I can complain as a member of this community.
I have the following problems with the new way of getting XP:
1)I can't estimate how much time it will take me to get the xp I need, to meet my daily goal. Until now there was an almost constant xp/time formula. I knew that I needed around 30 min daily to meet my xp goal and I programmed my day according to that. Now the time I need to get the xp depends on how many reviews I have and need to do.
2)Many times until now I felt the need to stop moving forward and keep reviewing for a few days, and I could still compete with my friends and stay at the same level with them. Now either I should stop reviewing or stop competing. I fail to see how, pushing me to finish my tree faster, will help me become fluent in the language.
3)I m in Duolingo cause I like the fact that here I can move forward at my own pace and set my own goals. People here don't review to get xp, they review because they feel they need it. If you treat adult learners like spoiled kids that need a push to study forward you are in the wrong boat.
4)I m very annoyed that teachers here are getting a special treatment. Of course If you try to implement a system like Duolingo for schools, kids will find ways to abuse the system.Why should anyone change something that works for 90% of the users to benefit a small number of teachers and kids.Teachers, If your students are not motivated enough to want to learn more, it's not Duolingos problem its your problem and you do a really bad job.
5)I feel that there are many things that could change in Duolingo and really would help. For example there could be more lessons to buy with lingots. New banners to unlock with xp or lingots. New outfits for Duo.Or maybe more complicated addons, for example time competitions with strangers, pay 5 lingots to get in the competition and if you finish first get 10 lingots. The thing we love in Duo is gamification. Go back to the roots of Gamefication and think of ways to make it more fan and engaging. Reducing the xp or doing changes like that only creates frustration and i dont think it helps the community.
I really respect you, Dr. Luis and Duolingo team and I m really thankful for your work, but I think Duolingo's orientation to schools and teacher is wrong in my opinion. Adding a bunch of kids that don't want to learn and a bunch of teachers that see Duolingo as the best way to make their jod easier, is not going to work. Changing Duolingo to better suit the needs of those teachers will not make it better but it will slowly destroy it.
I m strongly protesting, and I will set my daily goal to 1 xp. In that way I will not let the change affect me in any way and I recommend that everyone here who shares the same opinion, to do the same.
And in order to be proactive, I already have read your comment, Dr. Luis to another Duolingo user:
"I'm not sure what you'd like us to do. Go back on an experiment that increases engagement? We're working on improving everything, and yes our strength model will get better over time."
My argument to that is that many users at first will try to keep getting the XP they had set as a goal before the change, so the numbers will look great. But will they keep doing it for a long period of time? I would say that in my experience if you push someone to the limit he is not enjoying it anymore, he will just quit. Right now I see Duolingo team as the caring father that is telling us "its good for you, trust me." and we are the kids that say "..but we don't like it." The problem, Dr.Luis, is that we are not kids and even if we trust you, we still don't like it.
I hope you will take into consideration my opinion. Ευχαριστώ για το χρόνο σας. (Thank you for your time.)
Sincerely, Chris Giokas, a Duolingo fan.
(If you agree with me upvote this post, in order to make Duolingo team and Dr. Luis see it. If you notice any mistakes, please let me know, my native language is Greek so it's not easy to write a long post in English with zero mistakes.)
<h1>Agreed! To support this idea, I would suggest that you add: (I understand if you don't want to) :</h1>
The teachers/students should have a separate interface that does the following:
Restrict students from visiting the Discussion forums and Lingot store (the Lingot store can be used for cheating and avoiding assignments, so it might as well be removed. However, teachers can "award" items from the Power-Ups section in the Lingot store to students for good behavior if they want.) Bonus Skills cannot be assigned, 'cause, you know, "Flirting"... The items in the "Practice" section of the Lingot store are available to students without purchase.
Students are encouraged to use the "Strengthen Skills" button as often as possible to retain what they've learned.
"Homework" can be assigned in the form of skills or parts of skills. After the "homework" is finished, a grade is automatically calculated based on how many answers the students got correct.
Teachers and students can use an Edmodo-style platform instead of "Discussion" to communicate with each other.
When teachers first sign up, they are given a guide/tutorial that shows them how to properly use the educational part of Duolingo.
Immersion is not available to students.
The Coach is not available to students. To be honest, the students' "coach" should be the teacher themselves!
The teacher assigns what language the students can learn, and the students can't change or add new ones.
An exam can be given to the students in the form of a Progress Quiz, but can be fine-tuned to fit only specific skills.
Generally, the educational section of Duolingo is focused on how many activites you do, such as skills or strengthening of those skills, not how much XP you get.
As a former teacher? I shudder at the thought of teachers being generalized as lazy because, honestly, many of us simply don't have the tools or the support to do our jobs the way we'd like. That said, the problem here is that the onus for teaching needs to be on the instructors, not on Duolingo itself. The original program would have assisted in my classroom a lot more than the current version.
I would have had my students do a set lesson (or series of lessons) per week, leaving practicing up to them. They'd likely find the -format- of this fun, so I would put my focus on them completing lessons rather than trying to force them to practice.
It would be great to lock or toggle certain features, such as allowing them to purchase bonus skills. (Flirting would be something fun for high school students and not something I'd want for elementary). I would want them locked out of public discussions because getting a bunch of obnoxious students with too much time and too much resentment for having to do something out of class is NOT a good combination.
Knowing how much time certain tasks will take is a -bonus- for teachers. I would want to know exactly what each lesson would entail. If my kids would be getting random number of questions, I'd be less interested in using Duolingo to supplement my teaching: after all, I want to know how much of a nightly/weekly time commitment I'm asking from my kids. I would need to be able to predict it reliably.
The base program was wonderful as it was. Most teachers would have been asking for tools to help them keep track of their class, their learning, and maybe a couple of fun things for motivation. Most likely wouldn't have asked for changes to shorten lessons or change practice.
These kind of changes benefit far too few people at the expense of far too many.
Actually there was a post that requested reduced XP from a teacher that was referenced in the top comment on Luis' post. Here: https://www.duolingo.com/comment/6492401
I think the general tone of this community reaction is the Duolingo for Schools has had a negative impact on the community at large, and posts like the one linked indicate that there may be something to the theory that changes are being implemented at the request of teachers specifically.
This x 1000. Also, for goodness' sake weed out the pathetic sexist gags and silly innuendos from the lessons if you want schools to use it. Get your heads around the fact that schools are subject to regulation (and basic professionalism) which prevents them from exposing kids to this stuff in the classroom, and that schools have to promote cultures of respect and equality. There is no point in anyone worrying about the 'flirting' section when they're all going to have to do the main French lessons.
I agree. Ever since Duolingo has been introduced to schools, there have been tons of little children spamming the discussion with random spam or "moar lingotz pls".
Evidence that they haven't mastered their own language yet. Sort that out before teaching them another one.
Random discussions are nothing new. The reason why it seems worse might be that people now have the schools feature to blame for it or because their tolerance is wearing out.
Yes but I have never seen them at this extent where almost every other discussion is spam
I think it is because there is a large influx of new users at roughly the same time. If (these are purely made-up numbers) 2% of new users are going to be random spammers, and there are usually 100 "new users" at any given time but now there are 10000, the percentage may not be any higher, but the absolute number is, and that makes it more noticeable.
(It also may not help that the students using Duo for Schools can't just go away and stop using Duo.)
It's the sheer volume of the spam. It seems to have grown by at least 500% in the past few months, to the extent that it's almost impossible to find any meaningful discussions in the forum.
I find it annoying too even though I am technically a student(i am learning outside of school) and it gives all students a bad rep.
I agree the immersion feature should not be available to students. There are enough mistakes made as it is. If someone corrects me and they have a better translation, I'm happy to learn from my own mistakes...high school students are going to bring a LOT of ego into the ring so I would ask we anticipate nipping that in the bud as much as possible before it becomes a problem.
I think your comment and point 4 in the parent submission speak to the different needs of people who are operating under external motivation (i.e. instructed to use duolingo) versus people operating under self-motivation (i.e. use duolingo because they want to). The two learning environments are hard to reconcile and should possibly be separated (as you are suggesting).
For real your comment hits the nail. "The two learning environments are hard to reconcile and should possibly be separated (as you are suggesting)." with one phrase you got exactly the heart of the problem. Have my lingot Sir.
I don't use Coach, so I don't care about xp in terms of that, but it is very annoying (i.e., not rewarding) to spend a lot of time and 46 or more tries to freshen a skill that I obviously either didn't get in the first place or really, really need to review (no matter how 'weak' the system thinks it is) and only earn 3 xp.
I have completely moved to 'timed' review even if I'll have to review more frequently because of this particular change. This change and the addition of coach continue to get a thumbs down from me.
I agree RootAnn, I've never used the coach and thought it was a useless idea, frankly, I still don't get it. I don't care about xp either as I am already at level 25 and can't go any higher. What I do wish though is that Duo finds a way to help us practice on words/phrases that we constantly get wrong or struggle with. The strengthening exercises are useless when they keep giving you the same words/phrases 3-4 times in the same lesson. How many times do you have to translate "le raisin" or "il doute" in a lesson anyway?
I agree with this so very much. I did a generic "strengthen skills" exercise in French last night (because I review daily, and all my completed skills were gold). There were 2 or 3 sentences I really, really did not get. I needed more repetition on those before I would have had a hope of remembering how that worked. (Ah, subjunctive, how I sometimes hate you.) Did I get those sentences more than once? No. But I got at least a dozen sentences practicing the word "juillet."
Hi Maggie, I have noticed in the past few days that someone is listening to us. The strengthening exercises have been much better. I have been getting more words/phrases that I need to practice but there are still repetitions. I think they're working on it. Let's keep our fingers crossed.
That isn't happening for me. (I don't mind getting repetition if I missed a question. But 12 different sentences all practicing "la jupe" happened in my review this afternoon.) I think part of the problem is that it's based on "weak words." If I'm weak on "la jupe," a couple of repetitions is enough to remind me, "Oh yeah, that means skirt." If I'm weak on "j'aille," there's a good chance that I recognize it as a form of "aller" and can guess the meaning, but I don't know/understand how and when it is used, so a couple of repetitions will not be enough.
I see. I guess it's different for me because I am not as advanced in the lessons as you are. I am still in the beginning of the tree (I've only completed 23 lessons - infinitives) because I complete a lesson and then strengthen until I am comfortable that I know the lesson. That's how I am already at level 25. I am afraid to move on to adverbs right now because I keep getting little things wrong, like a, en, de, etc.
Thank you for this letter. I really appreciate your passion, and my hope is that our team -- the same team that created everything you've been used to so far -- can create the best possible website for you.
I think it's worth saying a few things:
This change was not done because of the teacher dashboard. We try new things all the time, and now we're trying to encourage people to use Duolingo to learn most effectively. We do not think redoing Lesson 1 of Basics 1 over and over again is the most effective way of learning, so we wanted to start discouraging that. That's why we tried the experiment in the first place. And then it turned out that, based on our measurements, people actually liked the new scheme more on average!
We personally like the idea of rewarding more points for things that our system believes help you more, and it also turns out that the usage patterns of millions of people reflect that this new scheme is better. So, it is hard for us to go back on the decision.
Every now and then some community members think that we don't analyze our results well. In this case, the criticism is that if all we're looking at is the number of practice sessions people do, then obviously there would be more of them done because each one gives less XP. Yes, we thought of that too, so that was not the only metric we looked at -- in addition to more time spent on the site, a larger fraction of people also return to Duolingo in subsequent days after signing up or after having used it last.
I obviously don't want you to stop using Duolingo, so I encourage you to give the new scheme a fair chance.
All of this said, over the next few days we will try an experiment in which people get a few more XP for practicing. I'll keep you posted.
With all due respect, the community has spoken, it seems from the "statistics" shown on both the announcement and on this letter the majority of those interacting are against these new changes. Numbers are all fine and dandy, but the problem is that this community is made up of people, not numbers. We are individuals, we are unique, and acting as though you can discern our feelings based off a glorified spreadsheet is frankly quite offensive. We have given duolingo more than numbers now, yet we are ignored, we have given you personal thoughts, in the form of words, articulated by real human beings, not some computer software. Please don't get me wrong, I respect you and what you have created, and have been inspired by this idea since hearing your first TED talk. But when you start treating this community like big business treats their customers, it honestly takes away from the idea that you care about people, not numbers. We are duolingo, we are people, our thoughts, feelings, and educational ability cannot be summed up by statistics, and I would have thought that the outpouring of comments would have convinced you of this fact, yet still, more talk of statistics. Well let me give you one more, this time a simple one, in the 48 hours since the original announcement post it has received a total vote of 46, whereas the comment speaking against this feature is presently holding a vote of 378, not to mention the fact that people have given over ten times as many of their hard earned lingots to this poster than the original announcement. Since you like numbers more than the words of this community please mull over the possible reasons for such an odd statistic to occur on the announcement of a supposedly good change, you may even feel led to actually read what some of the humans behind the statistics have to say, our thoughts may surprise you. As for me, I figure I'll go back to my paid app, yes it sucks, but the business actually listens to people, and the funniest part is that this business that listens takes our money and calls itself a service, whereas duolingo and it's numbers is free and calls itself a community. Funny how things go.
Signed, another number on your statistics.
Thanks for creating DuoLingo. I've been wanting to learn languages for decades and this is the first time that I'm actually doing it.
I completed the Spanish tree in December and started the Italian on January 1. I continue daily review of Spanish while learning the Italian. The Italian needs a lot of review, because it's so new to me and because it's easy to confuse with Spanish. I have actually done very little review of Italian so far, rushing through the tree instead, but yesterday I decided to take a week to slow down and review. I don't review because it's easy XP. I review because I feel I need a 2nd look at what I skimmed over a month ago.
For me, the new XP system is a disincentive for refreshing lessons. When I refresh a lesson I like to do it at least twice to reinforce the relearning. Yesterday I refreshed an Italian lesson that was not golden, and got only 4 xp. What a letdown. I did it again and got 3 xp. I received only 7 xp for doing 2 lessons instead of the usual 20 xp. I have my Coach set at 30 xp/day. I need to depend on 10 xp per lesson, because some days it's all I can do to complete the last lesson just before midnight. If I can't depend on receiving a definite amount of XP for a given amount of work, I will have to reset my Coach to the minimum.
I hope the change in XP will not affect my study habits. I will continue to review when I feel I need it. I clearly do because my Spanish tree has 18 skills not golden (of 67 skills completed), and the new Italian tree already has 15 skills not golden (of 31 skills completed). My plan is to set my Coach to the lowest number possible (1 xp) and ignore XP altogether.
Why not be more transparent? If the data backs your theories, then post it, engage and share, don't hide away. I think that the data could be interpreted in other ways, because I would love to know, you you measure enjoyment (as stated in another post)? I can see measuring engagement, but people can be forced to engage (students) and can be frustrated while engaging. I think more time will show the true impacts of this change.
Since my post is being referenced here, let me clarify a couple of things.
- Personally, I find XP to be rather useless, but I recognize its value as a motivational tool in a gamification scheme. I worry that these penalties will negatively affect users for whom points are meaningful and as a result, harm Duolingo.
- This change serves to highlight some existing issues with both the way Duolingo estimates a user's mastery of vocabulary as well as how it chooses which words to review (it does a mind-bogglingly terrible job of actually choosing the weakest word forms within the lexemes that it targets in reviews).
- If not for these problems, I wouldn't have much of an issue with the XP change in principle.I do think penalizing the undesired behavior sends the wrong message, though, and it would be better to reward the desired behavior instead (Luis says they're about to test this).
- While I'm skeptical of a result obtained in only a week, I strongly advocate Duo's AB testing approach in general. When it comes to issues like this one, I tend to trust results of quality tests more than threads like these because...
- ...The number of up/down votes isn't particularly meaningful (though they're personally gratifying). While user feedback is important, it's critical that we do not overvalue it. A small subset of Duo's ~15 million active users use the web site (85% of users use mobile according to recent statements by Luis [at ~6:00 in the video]), and a tinier subset still use the forums. We may not be a particularly representative sample because the type of people drawn to the forums might be more prone to be bothered by a change like this.
The second and third points above are things that Luis and I discussed a bit elsewhere, and he acknowledged that these are issues that need to investigated and improved. If nothing else, hopefully this whole brouhaha will serve to get some much needed attention focused on the way they model strength and select words for review.
I would like to point out that I agree with much of what you are saying. If the xp were more fairly awarded based on actual deprecation of word strength, I would be all for this update. But I know for example my adverb knowledge is weak, but when I reworked the lessons my xp was 3,4, and 5. Now if it gave me 7, 8, and 9...I would be more understanding. I know there is a better way of doing this.
First the system should have a way of keeping track of a lessons distance away from where where you are in the course. Obviously newer topics with newer words are not set in memory yet. Of course we don't want people reviewing Basic 1, 50 million times...but conjugations should of verb 2 present...that should be reviewed often in my opinion.
Second, there needs to be a floor on when words are up for review. Like if it has been three weeks...it needs to be reviewed (or go by word strength, if the strength of a word is at 0.2...it automatically up for review next time you use strengthen skills).
On your final point, this is an issue with all organizations and companies. The vast majority of users of anything give no feedback (Apple users on the iphone? Windows users on Windows 8?), but let me point out that a large number of valid concerns have been raised and these need to be addressed.
For example, "a larger fraction of people also return to Duolingo in subsequent days after signing up or after having used it last."
This cannot be relevent. Your length of time for the test gives you no comparison to how many people leave the site or stop participating. From what I gather this test was a week (I was in the study group). Even if it was a month...that wouldn't be long enough to measure an accurate inflow and outflow of individuals from the site. Further, you don't even know what is the average length of time these individuals will stay.
Also, "in addition to more time spent on the site."
And what of us that have fixed time? I have, reasonably, and hour or two every night after the classes I teach end at 9:15. I cannot due 10 extra lessons to review, I just cannot. Nor am I ready to move on until I have my verb stems down.
This is why a longer test should have happened. I really feel like the word strength engine (for lack of a better name) should be fixed first. Then the limited experience will be more balanced.
On your final point, this is an issue with all organizations and companies. The vast majority of users of anything give no feedback (Apple users on the iphone? Windows users on Windows 8?), but let me point out that a large number of valid concerns have been raised and these need to be addressed.
I agree, which is why I qualified my statement, saying that feedback is important, but it mustn't be overvalued. A balance has to be achieved.
"With all due respect, the community has spoken, it seems from the "statistics" shown on both the announcement and on this letter the majority of those interacting are against these new changes. "
Thats not the truth. The majority has not even read this letter.
"the majority of those interacting" which would indicate that I was referencing the individuals interacting with the discussion threads.
Well let me give you one more, this time a simple one, in the 48 hours since the original announcement post it has received a total vote of 46, whereas the comment speaking against this feature is presently holding a vote of 378.
I think you're so interested in proving your point that you've forgotten some simple facts:
1) The count shown in any post is net votes, meaning that 10046 people could have votes for , and 10000 people against;
2) The most popular comment in that thread makes a lot of statements, including calling "xp meaningless". So it is anyone's guess as to what exactly people are voting for. The same applies to this thread.
Finally, if gamification is more important than learning for some users then I entirely agree that they should seek greener pastures.
And what exactly is my point? You've went out of your way to take one piece of my post and dissect it, but haven't provided any insight as to my general idea. I liked your post because you made good sense, and now you're picking mine apart trying to find flaw without looking at the big picture. Please make a point of your own if you will, but if you wish to speak to mine I ask that you speak to my point, not dissect it. You had a great idea, we are in agreement, so I have difficulty grasping why you felt such a need to critique my choice of words. The point about votes is still valid, it still shows amounts of agreement, because if the original post got enough downvotes to bring it lower than the reply wouldn't that still imply that less people were in agreement with it than it's reply? And if a true bit of nit-picking is in order, I was trying to make a point, that numbers don't mean anything compared to words, and the words are there. And as a last thought, gamification is what makes duo different, out of the tons of apps, duo is the one that makes it fun, that adds a community,and that adds a sense of achievement, otherwise we'd all still be using one of the dozens of other boring language learning apps.
I presume your point was that the community has spoken, and there's consensus that the change is not popular. But there are many flaws with that theory.
Firstly, Duolingo's user base consists of millions, not the handful that voted in that thread. If a proper study was done, there could be far more people liking that change than not.
Secondly, a very small part of the userbase uses the forum, an even smaller portion will ever comment or even vote.
As far as the change is concerned, in my opinion, this change simply exposes a possible flaw of the strengthen algorithm in which the weakest words are not always picked from the list, resulting in less points more often than planned.
If gamification is what makes duolingo fun, then adding more to it should make it more fun not less. The best and most addictive games are those which provide a hard challenge, and reward progress. In fact, Duolingo is based on an RPG model (Role playing game) which is notorious for awarding less and less points for redoing easy tasks.
A, duolingo has far more registered users than active members, please don't confuse the two. B, true, more may like than dislike, but I tend to think that the people who are active in the discussion are also the users that are more involved in this site in general, and in-turn are a decent representation of the user base, not complete no, but then again, most surveys are performed on a small sample group, as it is impossible to survey an entire population, and most of the time this sample group is cited as believing like the majority. And since I haven't mentioned it yet, the reason this change annoyed me in the first place is because it messed up my strengthen test, which now repeats the same material over and over without ever touching on my difficult subjects, which is counter-productive to my learning.
The fact that the people on the forums are the most active actually makes them a poor sample group for the userbase as a whole because it specifically excludes the (almost inevitably larger) less active users from being fairly represented, and their thoughts, feelings and experiences with Duolingo may not be accurately reflected by the most engaged subset of the userbase.
How on earth do you know a majority of users don't support the change, if you reject numbers?
Add me to the list of people who are fine with the change.
@Luis: Thank you very much for this detailed answer! I have a suggestion that you could discuss with your team:
It seems that most of the users who are active in the forums dislike this change because they now feel like getting penalized instead of encouraged. They gather 3 or 7 XPs for the same amount of work they have done before and that is not encouraging at all. You are right with stating that repeating Basics 1 isn't a great way of learning, but repeating even golden skills at the end of the tree is when you didn't learn the words correctly (even if the DuoLingo algorithm states something different).
Once you rewarded the user with up to 13 or 14 XPs for completing a lesson without any mistakes. That being said, what about changing your current 10/7/3 system to 13/10/7 XP? This way users are kind of getting penalized for repeating golden skills over and over again (this is less discouraging than it is now), but they are rewarded for excelling at lessons they have to repeat because of weak words (according to the algorithm - this is more rewarding than now!).
Putting it this way, you didn't change your new system at all - your idea remains! - but people feel less discouraged and more encouraged. Please take this idea into account and thanks for having read this.
The 13/10/7 XP system might be a psychological trick, but it probably works. Great idea!
Indeed - DuoLingo's whole reward and gamification system works with psychological tricks and rewarding tricks are much better than discouraging ones - that's why people dislike the great idea that much. It's not the idea itself, it's the implementation that isn't working very nicely.
I believe the main problem is the algorithm behind the "strengthen your skills" button. This morning I used it twice and it awarded 6 pts the first time and then 10 pts (notice that I have a bunch of weak words in my word list). It don't see the logic. I think that most people would be okay with discouraging redoing "basics 1" ten times in a row but that's a different story.
Luis, I just want to say, regardless of anything else, thanks for everything you do here and for making Duolingo possible. I hope you really do see the passion of the community as a good thing, as that means you have a community that really cares.
I absolutely recognize that we don't see everything that goes on behind the scenes, and speculation can get the best of everyone. You guys have all the data, and I trust that you do actually know what you're doing with it.
Finally, I just wanted to thank you as well for taking the time to communicate here on these discussions. I'm sure you know that community engagement has been a sore spot for a little while, so I just want to make sure you understand that I do actually really appreciate your taking the time to discuss important things in the community.
Thanks again for everything you do here and for making Duolingo possible.
So, at this stage is it safe to assume that the promises of new tests and "keeping us posted" was merely lip service?
Then why bother giving specific timelines? Simply telling the community what you think it wants to hear in the hopes that they give up and forget about the problem, and then going silent doesn't do much to help. It gives the impression that you're merely trying to placate users rather than actually address their concerns. One might get the impression that such promises are disingenuous.
You and Kristine have promised to be more open and to do a better job of communicating with the community since last year's furor over the web site's redesign - especially after your former designer's (naut/Marcel) public tantrum on the forum. If anything, openness and forthright communication has gotten worse since then.
On a related note, what exactly is the role of the community manager? It seems to be firefighting when there's an outcry by the community and occasionally linking an article that paints Duo in a positive light. Clearly you're not paying her to do nothing, so I assume she's busy with other tasks. Perhaps you should change her title to something that more accurately describes what she does.
As other users have voiced, I am skeptical that the statistics you have gathered on this implementation indicate unquestionable success. Yes, numerically it might appear successful if users are pushing ahead rather than repeating old lessons. As users, the majority (of forum posters, at least) appear to be expressing otherwise. A system that discourages review is frustrating, and I can tell you emphatically that the current algorithms for determining whether or not a user knows-their-stuff is inadequate.
Anecdotal as my comment may be, I suspect it reflect the sentiment of the community at large. It is off-putting to review a lesson I know I need to work on, only to have the system tell me through distributing xp that my time and effort was only 30-70% as valuable as doing a lesson I am certain I am totally unready for. My options are to ignore the xp system (which is a shame, because it was the competitive leveling system that enticed me to DuoLingo over other products) and proceed according to what I know works best for me, or to complete the lessons in a way that will look statistically like progress on your end, but is a waste of time on mine (otherwise I would have simply rushed ahead to finish my tree already).
Do you have any intention to post how, using the statistics you and your team have generated, you have reached the conclusion that the new system is optimal? It would be nice to open such things up to peer review (if you have already done so, my apologies).
I know it's slightly off-topic but just out of curiousity:
How do you analyse the A/B tests in general?
So how do you decide whether A or B is better/which group is progressing more, I always thought it would be about which group earns more XP or something but obviously that's not true.
Oh, and because I already started asking about Duolingos "mysteries":
What are the factors that make a skill "weak"? Just over time you didn't practise that word or also does it also calculate in error rate etc.?
And how are the words chosen that appear in a general strengthen skills lesson?
Personally, I think that you should go further not back. For example, 90% of the words covered in basics 1 are probably covered in most modules. So, it makes perfect sense to disable skill practice for basics 1 and basics 2 after they are golden and only re-enable them after they lose their "glitter".
Despite what some may believe overdoing something can and does result in negative effects, for instance a person who practices too often at the gym can cause irreversible damage to their body if someone doesn't prevent them from doing so. Perhaps the coach can be used for these purposes.
I strongly disagree. It would make perfect sense to disable single words - not whole skills! - and only if you really have the choice to! Why? Because Duo's algorithm is not perfect. In my tree, there are golden skills and I have forgotten at least on third of the vocabulary. Disabling them would be horror for me.
To be honest, I love DuoLingo as you can imagine, but after having finished my French tree, I changed back to Memrise because they do have a better spaced repetition system. You are getting drilled with words and their system is better for drilling single vocabulary. DuoLingo is way better for putting vocabulary into context.
Imagine you had the ability of disabling single words on DuoLingo or that you could create your own skill lessons only with the words you're currently struggling with (you can do that on Memrise Premium) - that would be the bee's knees! Disabling whole skills when they are golden does not necessarily mean that you know every word in it perfectly and that's why I dislike your idea (I still upvoted it because you're absolutely right with overdoing things).
> Imagine you had the ability of disabling single words on DuoLingo
We did, it didn't make a difference. Also I meant disabling skill specific practice, not disabling the underlying words. I don't think they should re-add it because users' judgement with regard to what they think they know isn't reliable. So they may end up disabling things they later forget.
Although few people know this, Duolingo does disable general practice once all words are strong.
Anyway, once a tree is completed I don't think a user should continue practicing indefinitely. At some point you need to let go and just use other material to practice.
I have had that happen (the disable general practice) and I can tell you, I didn't know all the words off the top of my head. With the mouseovers (and the fact DL doesn't care if you answer correctly), it is extremely easy to keep words golden, without knowing them.
Well, the fact of the matter is that even L1 speakers forget words. Sometimes a certain stimuli is required to trigger the certain memories. That's yet another problem Duolingo users have. The belief that they must master all words.
Duolingo definitely doesn't share this view, and if it did, it would only set a skill to golden when all words are 100% strong, which isn't the case, 80% is all you need.
Great idea, and a much better way to implement the original idea.
The act of punishing everyone for a few lazy people is one of the most asinine decisions I have ever had the displeasure of witnessing, and if they had done exactly what you just suggested it would have solved the original problem anyway. Yet instead they screw over everyone that used the practice feature to actually practice in favor of stopping a few lazy students. I hope duo will hire you, they obviously need some staffers with good reasoning ability and common sense, and you just came up with the best solution without getting paid for it. Bravo Dessamator, and please accept these lingots as my thanks.
In a way I don't mind too. I changed my daily goal to 1xp and the new change has no effect on me. But this change makes it completely impossible to set a high goal in coach if you have a busy life, cause you cant know how much time it will take to get the xp you need. So you make coach obsolete, and I really don't see why this is helping me. I see that helps lazy teachers that cant control their students and the fact that I have to change my way of learning to help those lazy teachers infuriates me.
Exactly! " I see that helps lazy teachers that cant control their students and the fact that I have to change my way of learning to help those lazy teachers infuriates me." That happens to me, too. I think that the best way to prevent this would be to separate the teachers and the regular users, so that the regular users won't be affected by the teachers.
I don't mind it either, but I also understand why some people are upset. Some people need a little assistance self-motivating, and the change was a major blow for those people. And the circumstances surrounding the change make it even harder to accept.
I get your point and I agree with you in some points.
People here dont review to get xp, they review cause they feel they need it.
I totally agree and that's the reason why I don't care about the amount of XP I get, so the change doesn't bother me at all. And the people you compete with have the same conditions which means that the amount of XP they get will also decrease so you can still compete review.
Based on data, not on opinion.
I always liked that the people on Duolingo want to learn, which created a warming atmosphere. There were a few trolls but they forgot about Duolingo sooner or later I guess. Not every student who uses Duolingo for schools wants to learn. And unlike other trolls they can't and won't leave after some time. I'm not saying that Duolingo for schools is bad, I just think that there are things to work on left. (Don't know how to solve that problem but I bet (at least I hope) the Duo team will find a solution.
On the other hand I can understand the fact that Duo wants to be an Education company and not a translation company and introducing the school feature seems a great step towards offering free education to everyone.
Well Duolingo is good cause everyone who is here now, wants it. No one makes us to be here. If you push a bunch of kids to do something they don't like, and lets face it not all kids are the same and not all kids like language learning, then you will get an army or trolls.
Our family homeschools our school aged children. My wife and kids have expressed some frustration with the new Duolingo XP change since their daily XP goals now require more effort/time to satisfy. I have encouraged the family to see this change in a positive light. I have reminded my family that our focus should be language proficiency instead of accumulating XP or racing to complete a tree before a certain level.
I believe the Duolingo team is attempting to correct a flaw within the XP system. The amount of XP and Level someone achieves does not appropriately quantify their language proficiency. This has been previously discussed on another thread (https://www.duolingo.com/comment/1276332).
To your point of "the school feature seems ... [to help offer] ... free education to everyone." Let us not forget the expression, "you can lead a horse to water, but you can't make them drink." Duolingo has been freely accessible to the world regardless of a person's age, background or location through the andriod, ios, and web platforms. Sure there are individuals who lack access to such technology and the internet, however the "school feature" in and of itself does not solve this problem.
Pretty good English Chris. Had no trouble understanding your arguments .. and I agree. Have a Lingot :)
The problem with the A/B tests is that they only go on for a couple of weeks at a time. A small change may well make the average user engage more just because it's something different, therefore it's almost impossible to see whether the change will actually be beneficial.
The recent change does not affect me in the slightest, however there seems to be many more unhappy than happy people with this decision which to me proves that the A/B tests do not actually measure what they should be measuring.
They're also highly skewed towards mobile users as well, which can put a damper on those of us who use Duolingo mostly through the browser.
Not really. They actually seem to do a pretty good job of compartmentalizing their tests. Most feature tests are handled separately by platform for the most part. In the current ab_options, 15 are Android specific, 33 are for iOS, and the remainder, save for a handful of tests related to lesson / practice session generation, are for the web. Though there are none in the current options list, there are usually a number of general mobile platform tests that seem to apply to all mobile platforms.
Just curious, where did you get your info on which tests are being run? I would find it fascinating to see.
I keep an (almost - one of these days I'll automate it) daily log of the changes to Duo's ab_options. While it's true that this particular test only ran for one week, that's very unusual by Duolingo's standards. Most run for months. There are currently a number of AB tests that have been going on since last September, and even a few that are older. They seem to typically run for at least a couple of months.
They had a problem with students and they had to solve it fast, by testing something for one week no serious scientist would take any decision. So considering the fact that Duolingo team are very professional I only can assume that or the results in one week were so amazing that it was brighter than the sun that the change must be done, or that the need to do that change to keep the teachers here was bigger than testing actually if its a helpful change. I believe that someone there said "look the metrics look ok and we have a problem we need to solve fast so lets do it"
I agree, but I was not making any comment on the quality of the testing for this change. I'm skeptical that there was adequate analysis in this case despite Luis's claim to me that the results were "clear and statistically significant." I was merely pointing out that the brevity of this particular test is the exception rather than the rule.
I understand exactly what you said. My comment was the explanation why this test was the exception and why they still haven't take it back. The test was the excuse to implement the change.
Ya, for all they know school started for students, leading to higher engagement, or conversely maybe vacation started for adults who do it in their leisure time.
I agree with everything here. I already set my xp to 1 when I noticed the change. Since Duo is not my primary learning source, I'd rather not spend the time just to keep a streak going. Also, I think it's great for teachers to recommend Duolingo to their students for extra weekend study/test preparation, but to use it as a primary course is another story.
OK surely they have to respond to something like this right? I mean I know they tend to ignore us users when we have complaints or criticize them but this user has fifty upvotes in less than an hour. People agree with him/her so DL should really respond to this.
Nice post. Hope you have also posted on Luis' thread that is a sticky announcing the changes. While I am a newbie here, I did know how much time it took me and now I have no constant idea (see your idea in #1) I am not a fan of the changes, like many others. I have found that it is slowing my progress and eventually will cause users to leave the site (just using my feelings as an example) Reminds me of a story: A town mandates that toilets be fitted to use less water. You are then having to flush the toilet twice as much to have the same effectiveness. So in the end, you use the same or MORE amount of water. To me, the information is like the water. More needed to get the same effectiveness. Change for the sake of change is not good in my opinion. By the way, your English is excellent.!!
I used Duolingo in school when it didn't have a school feature at all (September 2013), it was just the normal Duo, and it was so much better! All we had to do was work through the stuff and screenshot and print what you've done to make sure you've done progress! It worked perfectly fine, but now it's changed and it's a lot worse now.
Even though there are problems with what they did. Is it really all that bad? Just lower your goal to 1XP and you'll be fine.
The A/B test was way too short. 1 week. And even if that was long enough, a lot of the feedback I saw in Discussion was negative. I've already set my Daily goal to 1XP.
A great thing for Duolingo to have done would have been to have a survey just like the one that they had a week ago, asking about your Duolingo experience. That would have shown them the negative responses that people had toward the change.
Luis and Duolingo are not trying to ruin our Duolingo experience. On that I agree with you. I'm a big Duolingo Fan. They should also receive some praise for trying a rather risky move like this in order to improve our experience on Duolingo.
By the way, your English is great. Some minor punctuation errors and you'd be perfect! Keep up the good work.
Luis' post is down to 47 upvotes. The first comment pointing out all the flaws in the new XP change has 340 upvotes. That should make them reverse it immediately.
I already changed my daily goal to 1XP, but aren't you a bit sad that you had to lower your daily goal?
Why? I'm not emotionally attached to it or anything. We had higher goals and higher XPs. Now we'll have lower goals and lower XPs.
It's the same in Math, divide both sides of the equation by the three (which is what Duo did) and the result the same, the numbers just looks different.
I think if the XP awarded were more predictable/consistent, there would have been fewer complaints.
I agree and disagree... I think it has its advantages (the new system), but maybe if people could have a setting to choose? Because I, like Chris Giokas, have found myself needing to slow down lately.
And Chris, there were some mistakes but also very typical ones. And given that your native language is Greek, you did a good job... Very good job. The biggest mistakes I notice was just some missing apostrophes and one or two commas. Nice job, though! :)
I'd also mention using "costumer" which, in spite of the spelling, is not a mix of "customer" and "consumer".
Yeah, I saw other mistakes. I just don't feel like listing them out... It makes me uncomfortable.
A lot of the comments here are fair, but I actually think it is unfair to target teachers who use this as lazy, or that students using it should be treated differently than any other user. Let me first point out that I am a teacher of Spanish at the undergraduate level, and I have not successfully gotten -any- of my students to use DL. Take that as you will, the point is, not all of us require our students to use it, we just know it would be a wonderful tool to supplement their learning, which is exactly what DL is supposed to be anyway. I'm sure there are teachers out there who do come up with ways to make using DL more effective for their students while taking away the draw for them to cheat their way through it. Maybe a lot of the teachers who use it -are- lazy, maybe even a majority, but I still think it is an unfair comment against a group of people using this service just because they use it differently than you do.
mate you got me wrong here, I didn't say that teachers who use Duolingo are lazy. I said that teachers who have problems with kids who abuse the XP system and can't control them are lazy and not good at their jobs.
> I fail to see how, pushing me to finish my tree faster, will help me become fluent in the language.
It's entirely possible it would help. Humans are notoriously bad at judging how they best learn. If the Duolingo team has statistics showing that people do better this way than the old way, then it is entirely possible that they're more right about your learning patterns than you are.
A fat guy goes to a gym, and he decides that he wants to work out 30 min and no more cause thats what he likes and someone push him to workout more, of course it will be better for him, but if he doesnt like it and stops going to the gym, is it really helpful? Something is better for me if I like it enough to stick with it.
If 100 fat guys go to a gym, and 99 of them keep going even though they're being pushed a bit further than they planned, it might be a net benefit. The entire point of an automated teaching system like this is to figure out how to encourage people to do the things that make them learn faster.
Reminds me of my 2014 New Years resolution to start going to the gym. When I get a round tuit.
Unfortunately I doubt anyone at Duo's cares. The saint metrics have spoken, whatever that means, considering the fact most users are not going to leave the community just for that. But you're right.
I hate this system and I think it is incredibly stupid and ill-implemented. I just strengthened a weak skill and I got 5 points out of it. The skill was weak so it does look stupid, right ?
Besides, the revision system is completely broken. Now when I click on "Strengthen skills" I get a 2-sentence practice. 2 sentences over, and over, and over again, till the end of the practice.
Well, they won: I'm planning on finishing my trees and never setting foot on Duo again, as practicing has been made nearly impossible.
Hi Duolingo Team,
I personally also am no fan of the current program.
This is because the idea of what you are trying to achieve is not well connected with the implementation of the instruments you are trying to achieve this goal with.
A concrete example is the "Strengthen" Button. It should find weak words by definition right? Well it also only granted me 5 xp. And this is even though I have spent a long time in the third section of the tree without redoing any lessons from the second part - so there must have been some weak ones... There needs to be a way to know one can achieve a consisten flow of xp.
Also the indicators that measure "learning success" are arguably weak - to follow up on the more scientific discussion earlier: they seem to have a low internal validity.
And the point above connects to the argument of correlation versus causality: indicators with low validity almost always lead to false interpretation of data and causality, so the comment that said the argument was technically true but practically obsolete: it is in this case not practically obsolete.
And even in the sciences there are many many many debates about professional experimental setups, so solving this problem on a meta-level is not possible because the devil lies in the details no?
Why not make a poll with some advertisement and present the descriptive data to the community? You might argue that participation rate might correlate with "level of anger" etc but it should still (with big enough n) give a better and more satisfying picture for the community wishes and feelings.
Also you not only receive valuable data for your business model (whichever that is) but give the community a feeling of participation and expressing their feelings - this alone might calm some heatened spirits.
2 birds with one stone, if youd ask me.
"My argument to that is that many users at first will try to keep getting the XP they had set as a goal before the change, so the numbers will look great."
Exactly. If we do not "engage" by doing more, we will not reach our goal, and will lose our streaks as a result (I suspect this is what happened to me (staff has not yet commented), and the loss was 420 days, still not reinstated over a week later...). While I'm not overly obsessed about a streak number itself, it was mine and it feels as if it was taken from me. If this change caused its loss, I feel that it should certainly be restored.
I, too, have made the goal change to 1 XP/daily. How's that for "engagement?" Pretty sad, actually. Wait and see if they remove the 1 XP option. The app already breaks that setting for me, resetting it to 10 each time I use it, and thus, I will no longer use the app.
WHY YOU HAVE TO CHANGE ALL DUOLINGO????
WHY DON'T YOU JUST CHANGE THE DUOLINGO FOR SCHOOL PART AND NOT ALL THE WEB?????
Chris, At this writing (8 hours after your posting) there are only 127 up votes. You need several thousand users to upvote your letter before Luis and his team pay attention. Thanks for writing your letter. It reflects the opinion of at least 127 people. Good luck with your German studies. You have certainly mastered English well.
You are wrong, right now this post is the most popular in Discussion. Believe it or not, even the most popular posts of all times didn't get thousands of upvotes. So I m pretty confident that with 130 upvotes in 8 hours and possible at this rate around 390 upvotes in 24 hours, Duolingo team will see it and take the message that most users are not pleased. And thank you for the compliment.
I agree with you, Talca. There are millions of students on Duolingo. My heart bleeds to see that on Luis' post every comment who dares to be (even a bit) in favour of the 'new system' is immediately downvoted... (It's why I post my comment here, not there.) And like all social studies can tell you, everywhere there is a 'silent majority', who is not reacting, because or they agree, or they don't care. So be carefull with conclusions 'that the majority is against something' (or in favour). This is typical for social media (like Twitter) where people seem to think always that they speak for everyone, while they are only giving their own opinion (who can be shared or disagreed by others).
For me, frankly, the discussion is held on the wrong matter. I agree with some arguments of the 'pro' and also with some of the 'con'. The real matter in my opinion is that the XP-points are used to determine your level in a (learning) language. If you can go to level 20 simply by adding some a's in an immersion text, or doing the same skill over and over again, like some kids did, then there is something fundamentally wrong. So for me there is an easy solution (easy to state, I know, harder to implement) : let XP-points be gained by (meaningfull) experience, whatever it comes from (new skill, review, strengthen, immersion, and every new possibility to think of...), and let them count for your streak(goal), but find a new way to evaluate your level in a (learning) language, perhaps based on the (strengthened) skills you finished...
Why the "silent majority" is silent? If what you say is true why this silent majority is not downvoting the posts that are against the changes. As I see it there is a majority that didnt like the changes and there is a part of the users that dont care or are skeptical about it and havent decide if they like it or not yet.
People, in general, don't downvote as often as they upvote. They seem to interpret "upvote" as including "I agree with this message", but "downvote" is limited to "this person is not contributing to the conversation".
First : because - from that millions of students on Duolingo - not everyone reads (all) the posts (in all the discussion groups in any language...).
That's why I said Be carefull with conclusions...
Second : because maybe they use down/upvoting in a different way than you do (I upvote mostly only if someone says he or she finished a tree, or if - in a large discussion - I will bring an argument up in the list of comments, or if someone develops a really heartwarming argument about something; and - freedom of speech - I downvote mostly only rude talk or troll posts...).
That's why I said My heart bleeds...
First of all there are millions who joined Duolingo and never came back .The real number of active users is much lower. Also it doesnt matter if only a few of the users read the forum, they are enough big sample to know statistically what all the users may think. I don't see whats your argument, I would like to see an argument about how and why you think the new system is helping you. Its not a philosophical debate, we have a change and we comment on that change. In my opinion that change is not going to help me, and also It makes my learning routine more difficult so I didnt like it. I m not against anyone who likes the new system and of course I will suck it up and continue learning, but my arguments are valid.
No - what matters is not merely the relative size of the group, but whether the group is representative . If we had the views of a randomly chosen sample of duo users then that would be one thing . but the people who comment on the discussion boards are not a random sample, and thus one can't generalize from what they think to what the population of users as a whole thinks. This is sometimes why polling goes wrong - if a poll relies only on landlines, for instance, it will skew older, since more young people than older people have completely replaced their landlines with cell phones. If you rely on internet polling, your sample might skew younger. Pollsters work long and hard to make sure they get representative samples - but there's no reason to think that people who write into a duo discussion are representative of all users.
As for the issue itself, I don't think I care anymore (though I was not pleased when I first saw the change). I don't care about experience points that much anyway - I try to do a certain number of exercises (both new and general strengthening) a day, and that isn't affected by how many points I get. If it turns that I have to lower my goal in order to make sure I maintain my streak, well, then, I'll lower my goal. (I do care about that blasted streak!) But that won't affect how many exercises I'll actually do - that's determined by how much time I have, how much mental energy, how much else I have to get done....
The majority of the reader and voter of your letter. But 200 people are a very small group, comparing all Duolingo-Users...
I hope Duolingo gets better with the time, and the only way to get that is to make changes, i like the new Dou, and i think part of the solution is Dou can be more customizable in order to make everyone experience the best experience. That is my opinion.
I'm sorry, but in case you are not aware, the teachers don't have control over what their students are doing on this website, and calling them lazy and trying to blame this all on them is ridiculous.
I don't think lazy people can be teachers. It is a very difficult job. We don't like the way the pupils behave on the forums, just imagine having to deal with them all day!
Lazy people can definitely be teachers. They can't be good teachers, but they can be teachers. (Former teacher here. I've known many dedicated and excellent teachers, and many more competent-but-not-outstanding teachers [myself included]... but I've also known a few lazy and terrible ones.)
Agreed one-hundred percent! And your English is much better than most native speakers I know, so please don't be so hard on yourself.
I honestly think you have some good ideas here. I strongly protest about these topics as well, however, I think that this topic might be getting just a bit too controversial... am I agreed with here? We all have slightly different standards of what we expect Duolingo to be. We can't all have them all at once, which brings me to the students. Duolingo is, in fact, made for students, young and old, you and me. I do agree with your statement about the bias towards students... Overall, a great statement that Duolingo needs to listen to.
apparently Pearson is now a player or a payer to duo and as it is well known in the education world, Pearson is a plague to education.
chrigkio, I agree with your post and have already reduced my xp to 10 a day from the maximum 50 xp, to protest against this change. However, I don't think they are listening as they are still continuing with this change, though a lot of people have complained about it. I spend less time on duo now. I used to spend sometimes a couple of hours as I used to like the lessons and the commentary and earning xp to reach level 25 in a language. It is a shame really as I used to love Duo and would recommend it. Not any more.
PS, Your English is very good. I wish I could speak Greek just to say hello even lol. As for mistakes, I think you spelt gamification correct the first time and added e to a later spelling, and added a d for job. Those are the only mistakes I found, but I know if you it again you'll spot them. We all make mistakes, as no one is perfect. I'm sure I've made some mistakes in this post as well,Thanks for your post. I hope Duo team are listening.