https://www.duolingo.com/patfinegan

Suggestion: Paid opt-out of being an A/B guinea pig

Perhaps I am the lone malcontent in a forest of sybaritic followers, but I would gladly pay $20-30 per month to avoid being a DL A/B guinea pig. Please create a means for users who don't easily indulge disruption to forgo any and all changes to their version of DL until, say, 95% of the DL user base has been converted to the new system or upgrade. In other words, let us pay to be LAST in line.

The new French tree is brilliant but deserves only alpha or beta-test stature -- not because the moderators have been negligent or haphazard. Quite the contrary. They have been amazing. But until they receive months of beta-test feedback regarding acceptable English translations that they, for whatever reason, never foresaw, the course is not ready for prime-time. It is an incredible turn-off -- an agony I would gladly pay to offload onto non-paying customers.

March 29, 2018

7 Comments


https://www.duolingo.com/Dcarl1

The whole point of an A/B test is that it is a random group. Allowing self-selection undoes that. Surely kids wouldn't opt out (even for an adult $20-$30/month is a pretty penny), so then your test base skews less knowledgable, and less able to make choices about when and how to use it; then you can assume that the unenthusiastic or PT users wouldn't pay it, so your user base skews to a group much less willing to spend time on DL. And so on. This affects the use patterns, and therefore test validity.

I think the rampant, unfettered use of hundreds of A/B tests degrades both Duolingo's professionalism and its sense of community. More to the point, it gives the company a presentation more akin to a grad student test lab (that this program started as), than to a slick app of interest to VCs. It's amateur. I don't like the A/B tests. But clearly it is part of Duolingo's MO, and given that they aren't going to undermine its effectiveness.

March 29, 2018

https://www.duolingo.com/anomalousjack

I agree with your suggestion. I've long advocated a premium package for those who don't want to be part of an 'experiment' to improve the shortcomings of this project. $20 per month wouldn't be out of the question for me for that. But that product would have to be different - a professional, polished, and most of all, finished product.

Don't get me wrong, I don't mind offering some help to improve the free product for other users and I help a lot in the discussions. But personally for me, the 'testing' part is getting counter productive to my primary goal here. At last count I had 82 positive replies from course contributors for submitting answers through the 'my answer should be accepted' route, most of which seemed fairly obvious translations to me. On that evidence it might seem that DL is rolling out courses with only a couple of answers attached to each question for the express purpose of 'stimulating' their A/B testing agenda. Meanwhile, each suggestion I submit supposes I have made an 'incorrect' answer (albeit a fairly obviously correct one it seems!). All this is eating into my time and is simply unprofessional practice and now I'm getting pretty tired of beta-testing this half-finished 'experiment'. I feel like I'm doing someone else's job rather than learning a language!

March 29, 2018

https://www.duolingo.com/nskutt

As I was recently informed, you are in dozens, or more, of A/B tests right now as the 'experimental' group. You just don't notice most of them. There are really two ways of looking the A/B testing, one is that it's the best way of beta-testing because so many people will find the mistakes and it'll be that much swifter. The other is that if too many people dislike it than there's no point in fixing the mistakes. To find out about your A/B status check out this comment and just search (ctrl+f) for my username. Then just switch your user name for mine in the url after clicking the link and look at the "raw data" following jrikhal's advice.

March 29, 2018

https://www.duolingo.com/patfinegan

For the new DL-French roll-out, there's no need to check. It's obvious. 18 consecutive months of a completely golden tree reduced to a handful of shimmering branches. But yes, I do check my "experiment"/guinea pig status regularly. Thanks for letting everyone else know.

March 29, 2018

https://www.duolingo.com/nskutt

Thank you for your reply patfinegan. I just wanted add a couple things to my previous post. First, (I should have mentioned this earlier) this should be in the "Duolingo" forum and not "Troubleshooting" (see here for details on moving it). Second, this is why there is no response.

March 29, 2018

https://www.duolingo.com/JOAT2B

I support this but only if the opposite is also an option: I would like to be part of A/B testing!

March 29, 2018

https://www.duolingo.com/patfinegan

By all means!!

March 29, 2018
Learn a language in just 5 minutes a day. For free.