Duolingo is the most popular way to learn languages in the world. Best of all, it's 100% free!

https://www.duolingo.com/Rascalniikov

Why has duolingo made changes to the lesson progression without consulting users?

Rascalniikov
  • 13
  • 11
  • 9
  • 8
  • 8
  • 6
  • 5
  • 4
  • 2
  • 2

Why has duolingo effectively made the language tree format redundant by forcing players to complete the modules one at a time? What happened to the freedom to choose between 2-3 language modules at each stage of the language tree? I can't begin to explain how frustrating these sudden changes have been for me, since as a long-time user I was neither warned nor consulted about them. Please bring it back to the old format where users have the freedom to choose which lessons they want to complete, rather than pigeonholing them into a completely linear progression that takes the fun out of it completely.

2 years ago

52 Comments


https://www.duolingo.com/garpike
garpike
  • 25
  • 23
  • 22
  • 20
  • 19
  • 18
  • 18
  • 17
  • 17
  • 16
  • 16
  • 16
  • 15
  • 15
  • 15
  • 14
  • 14
  • 14
  • 14
  • 13
  • 13
  • 12
  • 11
  • 11
  • 10
  • 10
  • 10
  • 8
  • 7
  • 7
  • 5
  • 5
  • 2
  • 1099

If you look in your user file, you will see a whole list of things like this like:

leave_horses_head_in_users_bed_experiment":false

mix_salt_into_users_sugar_bowl_experiment":true

send_hired_assassin_after_user_experiment":false

tamper_with_users_brakes_experiment":false

go_through_users_sock_drawer_and_remove_one_sock_from_every_pair_experiment":true

And so on.

2 years ago

https://www.duolingo.com/CharmingTiger
CharmingTiger
  • 25
  • 15
  • 11
  • 11
  • 11
  • 7
  • 7
  • 2

... Brilliant.

2 years ago

https://www.duolingo.com/Danmoller
Danmoller
Mod
  • 19
  • 16
  • 13
  • 13
  • 12
  • 11
  • 6
  • 4
  • 2

Indeed.....

So many things to be fixed/improved.

2 years ago

https://www.duolingo.com/txredeyes
txredeyes
  • 17
  • 13
  • 12
  • 7
  • 3
  • 3
  • 2
  • 2

how do you find this? i've tried looking and i have no idea where that sort of thing is.

2 years ago

https://www.duolingo.com/YellowPlatypus
YellowPlatypus
  • 25
  • 25
  • 16
  • 6
  • 1202

Go to https://www.duolingo.com/users/4varice and run a search for "experiment".

I think it's at the very bottom of the page.

2 years ago

https://www.duolingo.com/IamBrazilian
IamBrazilian
  • 19
  • 13
  • 10
  • 9
  • 3

Do what YellowPlatypus said and then you'll find something like this: "sg_ltr_skill_order_experiment":true if you're in the experiment. You can press F3 to search these informations (if you're using Google Chrome).

Edit: ltr means Left to Right or something like this according to jrikhal

2 years ago

https://www.duolingo.com/Luis
LuisPlus
  • 20
  • 18
  • 9
  • 9
  • 4
  • 991

We will be fixing this today.

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

Hi Luis!

Thanks for weighing in!

Do you mean "stop this A/B test" or make that the lessons/skill made before the A/B test will not be lost? I mean the loss "I had 3 lessons done in the left skills then suddenly not only the right skill went gray but I went back to 0 lessons done in the left skill" heard on the forums.
And thus potentially loosing also the lesson done in the middle or right skill.

2 years ago

https://www.duolingo.com/Luis
LuisPlus
  • 20
  • 18
  • 9
  • 9
  • 4
  • 991

Stop for now. Then fix.

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

Thx for the quick answer!

2 years ago

https://www.duolingo.com/annika_a
annika_a
  • 22
  • 17
  • 17
  • 16
  • 11
  • 11
  • 5
  • 2
  • 2
  • 2

Thank you!!! :-)

2 years ago

https://www.duolingo.com/mjmc500

Hooray!

Thanks Luis - very much appreciated.

2 years ago

https://www.duolingo.com/stephenbal4
stephenbal4
  • 19
  • 19
  • 15
  • 14
  • 14
  • 13
  • 9
  • 7
  • 7
  • 6
  • 5
  • 5
  • 4
  • 2

Somebody's kicking themselves on your end for that bug.

2 years ago

https://www.duolingo.com/tara668
tara668
  • 25
  • 22
  • 18
  • 16
  • 16
  • 12
  • 10
  • 10
  • 7
  • 6
  • 6
  • 6
  • 5
  • 1380

jrikhal, I don't think any skills were lost. I did some exercises from a middle skill, found it had been locked and displayed zero, but after I finished the left skill, my middle skill displayed my original progress.

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

Thanks for the information about "not left" skills.
I only heard the issue about the left skill (in another discussion).

2 years ago

https://www.duolingo.com/tara668
tara668
  • 25
  • 22
  • 18
  • 16
  • 16
  • 12
  • 10
  • 10
  • 7
  • 6
  • 6
  • 6
  • 5
  • 1380

Thanks for the clarification.

2 years ago

https://www.duolingo.com/schmidzy
schmidzy
  • 25
  • 17
  • 17
  • 15
  • 11
  • 10
  • 85

Completely agree. The tree is designed so that you can do any of the lessons in the row you are on and they build on previous lessons but not each other. It allows for some amount of choice in what you are learning, and is particularly helpful in eliminating some of the drudgery of getting through the longer skills, since you have the option to take a break and learn something else.

This is an A/B test they are doing right now; not everyone has it. I sincerely hope the test will end quickly and we can all have our normal trees back.

2 years ago

https://www.duolingo.com/MultiLinguAlex
MultiLinguAlex
  • 25
  • 25
  • 17
  • 13
  • 10
  • 6
  • 5
  • 4
  • 2
  • 2
  • 2
  • 32

Exactly my opinion.

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

Why has duolingo made changes to the lesson progression

Duo didn't changed it: they're testing this alternative version (on part of the users, see below).

without consulting users?

They're not "consulting" by asking them directly because it's just not realistic to do so with so many users. So they "consulted" users (without users knowing) in the classical manner to do so in such cases: with an A/B test.
So users are consulted in a sense: the decision to change the system (=making it be for all users) or not will be taken depending on users' effective/real reaction (=change of behaviour) to the change (not on their feeling about the change).


N..B. : Personally (as an user and an incubator contributor(*)), I hope this A/B test will fail and this change will not go live for everyone.
(*) Contribs generally take this "row" system into account creating a tree... So planning to maybe change it without notifying them first so they can adapt their tree... :(

2 years ago

https://www.duolingo.com/flootzavut
flootzavutPlus
  • 24
  • 18
  • 18
  • 17
  • 14
  • 13
  • 12
  • 12
  • 12
  • 11
  • 8
  • 8
  • 7
  • 7
  • 6
  • 5
  • 5
  • 5
  • 5
  • 4
  • 3
  • 2
  • 2
  • 2
  • 2
  • 24

I am glad I've finished the trees I am most interested in; I like being able to mix and match instead of doing all the lessons in a skill in one fell swoop. Do you know how people who've finished a given tree are taken into account in A/B tests?

2 years ago

https://www.duolingo.com/Shady_arc
Shady_arc
  • 13
  • 13
  • 11
  • 9
  • 7

For me, rows were more like an impediment than something of any real use. Maybe good for skills that focus purely on niche vocabulary, not so much for skills that introduce grammar or more universal verbs and nouns.

2 years ago

https://www.duolingo.com/AislinC
AislinC
  • 16
  • 12
  • 2

Same with me. Some grammar skills are tough to properly understand and might take a lot more time to finish then skills focused on expanding vocabularly. Personally, I often grow frustrated if I progress very slowly with a long grammar skill, and those times it's really nice to be able to switch to another lesson. Positive feedback and all.

I really miss the option to chose from multiple skills - I hope they'll soon revert back to it.

2 years ago

https://www.duolingo.com/writchie4
writchie4
  • 25
  • 20
  • 16
  • 15
  • 1772

Honestly, the initial feedback for this test is pretty funny to me. I've always just done the skills in order left to right anyway and didn't even notice that I was part of a new test group until I saw it mentioned here. I'd be very curious to see whether most people on the site jump around between skills in each row or just do them in the order they appear.

2 years ago

https://www.duolingo.com/flootzavut
flootzavutPlus
  • 24
  • 18
  • 18
  • 17
  • 14
  • 13
  • 12
  • 12
  • 12
  • 11
  • 8
  • 8
  • 7
  • 7
  • 6
  • 5
  • 5
  • 5
  • 5
  • 4
  • 3
  • 2
  • 2
  • 2
  • 2
  • 24

I've always mixed it up, especially on longer skills.

Yet to figure out if I'm in the test group as I've been working on completed trees.

Edit: blech, yeah, I'm in the test group. I wonder how my actions or lack thereof will effect the test if I just neglect my unfinished trees? Anyone know?

2 years ago

https://www.duolingo.com/jrg635
jrg635
  • 25
  • 10
  • 9
  • 6
  • 5
  • 4
  • 2

I am apparently part of the test also. I would love to make more progress on my tree, but I am going to register my disapproval by only completing one lesson each day rather than completing a module or two as I would prefer. I guess I will just spend more time on the Spanish tree that I already finished than make progress in German.

2 years ago

https://www.duolingo.com/tjcinmaryland
tjcinmaryland
  • 25
  • 25
  • 25
  • 25
  • 25
  • 25
  • 25
  • 16
  • 14
  • 12
  • 9
  • 6
  • 1935

Which would, effectively, give them the feedback that I think you're aiming to deliver - that it's discouraging you from working on the site. I'm in agreement that it definitely discourages me, since I've often gone for more than one skill at a time, especially when there's a long skill on a given row....

2 years ago

https://www.duolingo.com/slogger
sloggerPlus
  • 25
  • 23
  • 23
  • 22
  • 20
  • 14
  • 1689

Right, jrg635. This is the most effective (just about only) way to vote against a change.

2 years ago

https://www.duolingo.com/DonFiore
DonFiore
  • 16
  • 10
  • 10
  • 9
  • 6

They are consulting their users right now. As a long-time user you might be familiar with the AB tests Duo has done previously (new German tree, XP to next level, amount of L1->L2 translations, progress bar etc.) and this is their newest one.

2 years ago

https://www.duolingo.com/GrandaUrso

I agree, I don't like it either. I like to leave the skills with the largest amount of lessons for last and suddenly it's the only one I can do! I want the old tree back just as much as anyone here. =) Hopefully this will not pass.

2 years ago

https://www.duolingo.com/territrades
territradesPlus
  • 25
  • 21
  • 20
  • 14
  • 12
  • 8
  • 8
  • 7
  • 6
  • 4
  • 3
  • 1746

I've seen a lot of threads lately were people ask how to progress, unlock the next skills etc. Maybe they want to simplify the system to avoid these questions.

2 years ago

https://www.duolingo.com/Sgt.Hammer
Sgt.Hammer
  • 10
  • 9
  • 9
  • 8
  • 5
  • 3
  • 2

I have seen those too. That could be a bigger problem than i realized, but,

Can we still call it a tree, if it is a straight line?

2 years ago

https://www.duolingo.com/stephenbal4
stephenbal4
  • 19
  • 19
  • 15
  • 14
  • 14
  • 13
  • 9
  • 7
  • 7
  • 6
  • 5
  • 5
  • 4
  • 2

By the technical Computer Science definition of a tree, it is still a tree :P.

2 years ago

https://www.duolingo.com/cswrawr
cswrawr
  • 23
  • 21
  • 20
  • 16
  • 14
  • 11
  • 10
  • 9
  • 8
  • 7
  • 2
  • 454

That's what I was thinking, seems like everyday there's a few people that don't understand how to continue...

2 years ago

https://www.duolingo.com/Fazac
Fazac
  • 24
  • 22
  • 18
  • 13
  • 11

This makes me worry for the world.... if so many people can't figure out how to progress through the tree by themselves, there is not much hope for the human race ;-)

2 years ago

https://www.duolingo.com/Sgt.Hammer
Sgt.Hammer
  • 10
  • 9
  • 9
  • 8
  • 5
  • 3
  • 2

They are changing the rules of the game while we play! I don't like it.

2 years ago

https://www.duolingo.com/trebach

It's Calvinball!

2 years ago

https://www.duolingo.com/CittaKyle

I enjoyed the feeling of accomplishment when I unlocked a new row (it would sometimes drive me to push a little bit further on a given day). I'm also one of those people who liked to mix it up a bit.

So... definitely not a fan of this change and would be curious to hear what anticipated benefits are

2 years ago

https://www.duolingo.com/amaslac
amaslac
  • 25
  • 25
  • 8
  • 7
  • 6

I don't like this. It'll make me do less and therefore progress slower. Some skills require more time and dedication to additional materials in order to pick it up right, and i'm not always in the mood or don't have time for that. In those cases I usually do other unlocked skills, this way I'll just do nothing.

2 years ago

https://www.duolingo.com/mjmc500

I'm part of the test group and I don't like it one bit. For one thing, a lesson that I had previously completed is now grey (it was on the right of a set of three).

I really liked the original format for this - if I had a "hard" subject, - for example Abstract Objects - I could split by learning with something more immediately interesting - Sports perhaps. Now i have no choice, and I am forced to slog through the lessons of something much less appealing.

I'm not even sure what they would be measuring here in the A/B test - it's not likely to make me use the site differently in the short term, as I have set myself some daily aims for the immediate future that require it. BUT, it is going to really annoy me and that could change my behaviour in the long run (i.e. longer term than an A/B test would run for).

How can they measure that? Frustration? Annoyance? (Honest question, if you know how Duolingo measures annoyance? I mean, we all know it's not by engaging with users in the forum!)

They already simplified the tree once a couple of years back - I really can't see the need to do this again.

2 years ago

https://www.duolingo.com/tjcinmaryland
tjcinmaryland
  • 25
  • 25
  • 25
  • 25
  • 25
  • 25
  • 25
  • 16
  • 14
  • 12
  • 9
  • 6
  • 1935

Typically, they register annoyance by seeing if it affects how much time you spend on the site, as I understand it. If a change negatively impacts the amount of time you spend, the number of points you earn, it tends to be 'negative feedback'.

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

it's not likely to make me use the site differently in the short term [...] it is going to really annoy me that could change my behaviour in the long run. How can they measure that?

It could: this could annoyed you so much that you'd not come back on Duo. Or just get you be bored sooner (than before, in average) each day and thus you'd take less lessons per day.
And some users (well, at least one in this discussion: jrg635) already said they'll only take one unique lesson per day to effectively show to Duo they are against this potential new version. So it's changing in the short-term the behaviour of some users.

I really can't see the need to do this again.

If the A/B test is positive, it'll show it was a good idea (thus the "need" to do it).
If the A/B test is negative (what I think and hope will happen), it'll show it was a bad idea.

They are exactly trying to find out (even if for me, as a lot if us, the answer is clear) if this is an improvement or not (based on real/effective users' reaction = change in their behaviour).

2 years ago

https://www.duolingo.com/mjmc500

I understand an A/B test, but this seems like such a minor change I can't see how it is likely to make people leave the site, at least in the short term - A/B tests don't run for long enough.

I am also aware of and have a lot of faith in "big data" and I suppose in this instance perhaps data on the scale that they are able to access will return greater/clearer results than I am expecting it will do.

And I suppose I should add that I also get the irony of me saying that this is a minor change that won't impact behaviour whilst it is also something causing me to complain about it. My reasoning being it seems too minor to alter behaviour in the short term but that doesn't mean that it isn't an annoyance and won't be a annoying in the long run. Plus, the layered tree now looks stupid.

Here's hoping that if they register no change in user behaviour, that they don't take that as a positive but that they return to the old method. They should ONLY make a change if it creates an increase in usage, not if there is no discernible change.

Oh and on a separate note, who runs an A/B test to a language learning site in January, when people have just set themselves New Year's resolutions that will have influenced their behaviour far more than any changes that they have made to the system?!

2 years ago

https://www.duolingo.com/jrikhal
jrikhal
Mod
  • 22
  • 18
  • 18
  • 14
  • 8
  • 5
  • 5
  • 5
  • 3
  • 3
  • 3
  • 2
  • 2
  • 2
  • 2
  • 2

I think it'll have far more short-term effect than long-term.

Either you weren't already on Duo before the test. It seems to me that in this case no specific long-term effect but maybe a short-term: the course is too hard for you (you can't pass a long skill and are thus stuck) then abandon (sooner than you'd have done with the other version). But one could argue that maybe with this version you're more focused (on each skill) thus go faster and thus you're less bored.

Or you come from the old system. If this new version don't get you to abandon/take lesson per day in the short term, I don't really see why it should in the long-term.

Oh and on a separate note, who runs an A/B test to a language learning site in January, when people have just set themselves New Year's resolutions

Because Duo also came back from holidays with new resolutions (like this one)? ;)

2 years ago

https://www.duolingo.com/mjmc500

Haha, too true!

2 years ago

https://www.duolingo.com/rachelrypkema

Isn't it good though, making you learn them in order, so that you can build new lessons upon previously learned concepts?

2 years ago

https://www.duolingo.com/Klaudialk
Klaudialk
  • 13
  • 13
  • 13
  • 12
  • 11
  • 8
  • 4
  • 4
  • 2
  • 2

The current tree structure already does that!

2 years ago

https://www.duolingo.com/rachelrypkema

Oh wow. I totally misinterpreted what he said. I though he meant you could do the whole TREE in any which order...

2 years ago

https://www.duolingo.com/akka13722
akka13722
  • 17
  • 9
  • 4
  • 4
  • 4

Also the way the trees are designed, it doesn't matter which order you do the modules in each row in.

2 years ago

https://www.duolingo.com/mizinamo
mizinamo
  • 20
  • 17
  • 16
  • 14
  • 14
  • 14
  • 13
  • 11
  • 10
  • 10
  • 9
  • 8
  • 8
  • 7
  • 6
  • 6
  • 4
  • 4
  • 4
  • 3
  • 2

At least if the tree is properly designed.

I remember a Turkish sentence which had a passive participle... confusing because the "passive" bit was taught in the left skill (which I had already learned and which was where I saw the sentence) while the "participle" bit was taught in the right skill -- so if I had done that first I still would have been confused :)

I think it can be difficult to take into account what depends on what prior knowledge, in all the edge cases.

In fairness, though, that was the only sentence I can remember where a sentence depended on multiple skills from the same row.

2 years ago

https://www.duolingo.com/slatour
slatourPlus
  • 25
  • 25
  • 20
  • 20
  • 16
  • 15
  • 15
  • 14
  • 14
  • 14
  • 13
  • 1243

After reading this post, I went to check out my tree and It turns out I am part of the test and did not even notice. I've always just tackled the next project in front of me and not worried about what comes after it. I guess I am somewhat single-minded.

2 years ago

https://www.duolingo.com/gingerninja3148

I don't really mind the change as I usually do it in order but would rather it go back to the original way. I know this sounds weird but seeing a coloured skill and a grey skill in the same row just doesn't look right. At least it will make me want to finish the first skill quicker so they can both have colour :) I also agree that sometimes I would mix up the order if I wanted to have a bit of a break and learn a fun topic or something. I liked the little freedom you had.

2 years ago

https://www.duolingo.com/marcpongnon

what do you mean?

2 years ago