Could the developers give the users a certain measure of control?
I was thinking about all these differences that users have in the interface and it keeps frustrating me more and more. For me as an user, having shorter lessons feels like learning less and i have no power at all to change that.
Now, i put my students to compete and i keep showing them the dashboard, but one of them has longer lessons with the old 3 hearts in his cell phone and complains that it is not fair. And it isn't! Had I control of this, i would discuss this with them during class and then set the same standard for all of them.
If you are a developer of this site and you're reading this, consider that giving the users a way to control the interface of the game can also be a way of experimenting: you will know which features are enjoyed and which aren't. After all, if your objective is to improve the user's effectiveness, maybe you should just let each one of us choose what we feel that works better!
Or you could at least set the same terms for all my students in the same group at the dashboard!
the idea of an A/B test (of a feature) is to chose randomly the reference group and the group having the feature to be tested. If you let to users the choice of the group they belong to, you'll have bias in your study.
Also, from what I understood, Duo relies on statistical automatic measurements of real "user's effectiveness" (and other measures), not on "what ones feel to work better for him/her". First because users can think something is works better for the learning when it's not the case. Also because some users will be more effective if challenged very hardly, other all the contrary. Some will be more effective with some features other not etc. So Duo needs strong, automatic and duplicable statistical tests to determine what are the ways to teach best to users in average.
I understand these principles, jrikhal. I just meant that the ease with which an user studies is directly proportional to the chances he has of not leaving the app/site altogether. Because i'm not allowed to grade some of my students with their duolingo progress, for instance, many give up. And their biggest complaint was the old heart thing that really frustrated them.
It's not about whether or not it really works better when it comes to learning: it's about having a flexible platform for students with different needs and profiles rather than considering them in average.
Of course, my younger students love to compete and for that they need to have a standardized interface, but many of my adult students could need shorter or longer lessons according to their needs and to the amount of time they have. If such things could be controlled by me rather than simply being used to test variables, I believe I could encourage them by providing a more customized experience. After all, time availability, motivation, and other sources of english learning affect the student and these things aren't accounted for unless you know the given student.
Hi there! jrikhal is correct about the spirit of AB testing - we have gained a lot of valuable insights from testing new features like that. However, we are aware that students who are sharing their progress with a teacher are a special group of users. After all, if using Duolingo counts as extra credit for a student, we need to acknowledge that that is a bias. We also want to try and avoid scenarios where students in the same classroom have vastly differing experiences on Duolingo.
We already take this into account when it comes to AB testing different versions of the skill tree, and we make sure that all students who are sharing their progress with a teacher get the same version of the skill tree. Moving forward, we are planning to find a solution that leaves students as unaffected by AB testing as possible.
However, the problem you describe (some students have hearts and others don't) is not, in fact, caused by AB testing. Hearts have been replaced by the progress bar on all platforms except Windows Phone, so it seems like your student is either using Windows Phone or an outdated version of Duolingo. Maybe you could find out what phone your student is using, and whether it's possible for them to update to a more recent version of Duolingo?