Duolingo's focus on usage metrics vs teaching effectiveness
It seems to me like Duolingo is too focused (almost obsessed) with A/B testing absolutely everything and then valuing usage metrics from these tests over actual learning effectiveness.
Don't get me wrong, I think Duolingo is a fantastic resource and gained lots of out of it so far, and have enjoyed myself while doing so. But it just seems lately that the decisions the site is making are based only on how it affects usage statistics, with language learning effectiveness being an afterthought.
I'm worried that Duolingo will only get easier from here on out as it focuses on increasing its audience, instead of focusing on how to take people even further in their language learning journey.
This is just my opinion and I mean no disrespect to all the people that I'm sure are working very hard behind the scenes, and they have created something truly great here. I just hope it doesn't lose sight of its original goal.
If you've read this far, thank you for your time, and I'd be interested to hear your own opinion on the topic.
Duo has repeatedly said they do not attempt to teach to profiency. I can't see how long term retention is even a concern with that stated. It seems more likely the objective is to build membership as large as possible to make the company attractive for buy out. Which is ok, they aren't charging me. I am free to quit when I want unfortunately in my case that's less than six months out. I won't be starting another language regardless of what they add and have finished both the tree and reverse.
They teach the basics, yes, and there's no point learning the basics unless they're retained. They've not only said that they aim for retention, but they've even published the details of how they do so:
How else do you find out what's effective, if not by testing it?
...but you have a point about retention vs effectiveness. That's always going to be a trade-off - on the one hand, people can't learn if they've left, but on the other, yes, sometimes something is lost in making the site more friendly to every possible student.
"Usage metrics" is more than simply the number of people who use the site, it also measures things like what parts of the site people use, how long they use them for, how often they use the site, whether there are differences between identifiable groups of users (do night-owls make different progress than early-birds, for instance), highlighting particular skills that cause an issue for users, and, when an A/B test is in place, how the behavior of the A and B groups differs.
It's a far more complex area than simply counting how many users log in each day.