https://www.duolingo.com/profile/Fuurinkazan.

NEW AB TEST:fewer lessons, less challenge, more pictures and multi choice, automatic crown level ups

With a heavy heart, I've come to inform you that the new AB test makes skills have fewer lessons, and some crown level have automatically leveled up as a result of this update. Shortly put, this is a dumbed down duolingo. For a second I thought I had typed www.dumblingo.com but no.

Why is it dumbed down? The low lesson count especially on the crown level 4 (100% typing, 50% in the UI, 50% in the target language) is what really takes the value and efficiency out of duolingo now. The crown level 0-3 and 5 mainly consist of multi choice, image and fill in the blank exercises, exercise types that do not require a lot of different skills from the language learner. Thus the possibility of doing a lot of lessons constructing sentences on your own in the target language which tests many different skills is now taken away from us. There is not much challenge left with this new set up.

Really hoping this AB test bombs so fast their head will spin.

It really took out my motivation to use this platform.

August 31, 2019

108 Comments


https://www.duolingo.com/profile/Cindy1100

They should have made it more difficult, not less difficult!

August 31, 2019

https://www.duolingo.com/profile/cmokba

Agreed, it's too simple as it is, no need to dumb it down any further.

September 1, 2019

https://www.duolingo.com/profile/DanielVanD18

Exactly. They should add more grammar skills, and actually teach grammar. I’ve noticed (I don’t know why) they try to avoid exceptions. I can’t think of an example, but I’ve noticed it before.

September 2, 2019

https://www.duolingo.com/profile/Semeltin

I've always said that. At least as an option. But Duolingo hate options and really want to appeal to people who are never going to become fluent.
Then Duolingo supposedly addressed this by adding crowns (consequently not as an option) which caused some problems. But after testing out the first two or three crown levels the difficulty was acceptable for new languages.

September 1, 2019

https://www.duolingo.com/profile/Nanase-Miyuki

I haven't got this update yet but I am angry that they are bringing this change, plus the premature removal of disable word banks from the apps. I am sick and tired of Duolingo continuously removing one useful feature after another and making the user experience WORSE EVERY TIME. And they don't even ask for user feedback.

Does it ever stop?

I feel like Duo keeps shooting itself in the foot every time these changes are made.

August 31, 2019

https://www.duolingo.com/profile/Algarve11

Yes, Duolingo is killing itself. I feel very angry too. The possibility of writing again and again phrases in your target language was the best they had - and now we're going to have less of it.

September 1, 2019

https://www.duolingo.com/profile/Nanase-Miyuki

They will kill itself even more when they give everyone the Health system and force people to uninstall the apps.

September 1, 2019

https://www.duolingo.com/profile/Algarve11

I guess so. Many users here hate the Health. And most of them probably work from the app.

September 1, 2019

https://www.duolingo.com/profile/uphilldweller

I suspect Duolingo are aiming to please the younger student. There is a balance to be struck between attracting (and retaining) new blood whilst not frightening the horses.

September 1, 2019

https://www.duolingo.com/profile/Semeltin

I suspect Duolingo are aiming to please the ones who aren't dedicated and will never get fluent.
Why am I assuming this? Because younger students have to put in the same amount of work to get anywhere. The ones that don't are those who always say they've taken four years of language X and can't speak a sentence. And by constantly making annoying changes Duolingo are driving away the dedicated.

September 1, 2019

https://www.duolingo.com/profile/wyqtor
  • 1719

Give them user feedback anyway. One star reviews on the Play Store and AppStore. I did my part.

September 1, 2019

https://www.duolingo.com/profile/GingerSquare

Yes, me too. Enough one star reviews will eventually get attention.

September 14, 2019

https://www.duolingo.com/profile/BeaverGuy

"Really hoping this AB test bombs so fast their head will spin."

That was what I was thinking while reading your text.

Is there an announcement anywhere for this AB test or a link/source?

August 31, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

I got this A/B maybe two weeks ago, and I agree that it's quite a change.

In an attempt to make the most out of the new system, I've gone back to a study pattern I was using when I first started on DL. It will look crazy to most, but is how I'm making the most out of the new hyper-short, low complexity crowns system.

  • I keep an index card for each of the sentence pairs in the lesson.
  • I keep track of every correct target language translation. (If I make an interesting mistake, I also track those.)
  • I try to never give DL a target language translation that I already know to be correct.
  • Since I'm learning French, I think about other ways of writing the content (changing questions to inversions, switching "they" between «ils» and «elles», practicing past participle agreement, using female variants of nouns («une voleuse» instead of always using «un voleur», etc), dictionary surf to see if I can find a synonym. Etc.)
  • These notes go on the index cards, with pre-prepared ideas to try the next time I get that translation.

The next time I encounter that translation pair, I find the index card and work on one of the un-tested ideas until DL is happy with it. I try to never repeat a translation I know to be grammatically correct/acceptable because it's a wasted opportunity.

For example, in a recent skill:

  • I had a total of 3 chances (yup, only three chances going from CL0 to CL5) to free-text translate "Did she find the dress that she was looking for?".
  • I answered differently each time (Elle a trouvé la robe qu'elle cherchait ? A-t-elle trouvé la robe qu'elle cherchait ? Est-ce qu'elle a trouvé la robe qu'elle cherchait ?)
  • Bottom line is that I try anything to maximize the amount of French grammar I can learn with each attempt.

I.e., in the new system, there are so few "type in target language" opportunities that I need to optimize the value out of each chance.

The good news is that I'm finding all of this hand writing on the index cards to be a decent replacement for the now extinct online repetition. Plus, I'm doing a better job of exploring alternate sentence structures and dictionary surfing.

Given DL's new structure, this will be my path going forward until things change again.

(I realize that some might consider what I am doing as cheating because I'm leveraging my notes. What I would say is that for me, this is an effective learning style and mimics the patterns I developed in college. I'm a low XP/day type and I'll always be in the bottom of the leagues, so from a competition POV, I'm the definition of a non-threat. To each his/her/their own!)

EDIT: Minor edit to give an example of what I'm doing.

September 1, 2019

https://www.duolingo.com/profile/Alyssia925801

It's never cheating if it works for you - the aim is to learn after all. I have a workbook where I write down all grammar lessons and word lists etc as this allows me to learn most efficiently. I also have some hard textbooks with additional exercises - it's not cheating if it gets the lessons into my thick skull.

September 1, 2019

https://www.duolingo.com/profile/uphilldweller

(I realize that some might consider what I am doing as cheating because I'm leveraging my notes...)

This isn't cheating; fwiw I've been making notes too. What I think would help is the student being able to mark a specific lesson for a second or subsequent run through. Instead we get a general revision option at gold which doesn't cut it for me.

September 1, 2019

https://www.duolingo.com/profile/Melyndi

wow that's a lot of work

September 1, 2019

https://www.duolingo.com/profile/Judit294350

Really sad.

August 31, 2019

https://www.duolingo.com/profile/Mereade

Really? I was hoping that taking a longer break would mean returning to an improved and more stable version of Duolingo. Well, if it goes like this, I am not sure coming back would be worth it (the Latin brought me back for a moment today).

I am not sure that this dumbed down version will be that unsuccessful. Lots of people actually take Duolingo more like a game than a learning tool, that is the main target public now :-( And let's not forget that should Duo get rid of the most useful exercise (the translation, especially to the target language), they will also need significantly fewer alternative answers, which saves some work and time. Fill in the blank doesn't allow that many variations of the right answers, the multiple choice or pictures do not allow any. They don't teach as much as the translation, but I am not sure Duo cares.

September 1, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

I suspect you are correct that part of the motivation behind the move towards even more fill-in-the-blank/mult-choice/etc is to reduce the activity/questions (and confusion) in the user forms.

Plus, it would make tree development easier because they don't have to instrument for all of the alternate translations, as well as back-fill as users file reports. From what I've experienced on the new "in house/staff" French tree - they may not have sufficient staff to cover these tasks, and may not want to change the biz model to bring all that work in-house.

September 1, 2019

https://www.duolingo.com/profile/Mereade

Well, then they should have kept the volunteer model or find a way to combine the two (such as keeping the volunteer team for adding tons of alternatives). My main problem with the in house courses is, that they are sloppy and not better than the volunteer ones. What can be easily and gladly forgiven to a team of volunteers is simply not acceptable from professionals.

The translation exercises are the thing that has kept me returning and using Duo for practice and review. If they are slowly getting rid of that, either to make the work easier for them, or to cater to the people who just want to pretend learning, it is really sad.

There are a few services already doing the fill-blank and mult-choice based exercises much better than Duolingo. They are using these options to their full potential (Speakly, Clozemaster,Kwiziq). But I still miss the full translation typing on those sites a bit. I have got a translation based workbook for some of my languages and it is awesome. It was just really comfortable to type the translations in Duolingo, and no competitor seems to be headed in this direction.

September 1, 2019

https://www.duolingo.com/profile/Mereade

just adding: thinking of it, I wouldn't mind the overall lowered amount of exercise. I think it was getting a bit ridiculous. Some time ago, I counted it. If you'd do the new Spanish tree in the suggested way (all the lessons without test outs, at the pace of 50xp per day), you would get to an incomplete A2 in four years or so. That's much worse than even an average language school.

The problem is the lower the amount of useful exercise. If it was up to me, the courses for the european languages (I am not knowledgeable enough about other ones to judge), could have only the Level 4. :-D That would actually be a good amount of useful exercise, I don't think the levels 1-3 would be too missed and anyone's learning would be poorer. But they are cutting down exactly the level 4, it seems. That is unfortunate.

September 1, 2019

https://www.duolingo.com/profile/JayHubert1

I agree with this 100 percent. Once the crown changes came around the thought of actually finishing a tree to 5 became too daunting. I’m at crown level 22 and just recently got to the third tier of lessons. I really enjoy the « free type » feature. That’s all that should be available once you get 5 crowns in a level. That would be a perfect way to review

September 2, 2019

https://www.duolingo.com/profile/Pancho_Delanave

My main problem with the in house courses, is that they are sloppy and not better than the volunteer ones.

I am sad to read that. If they are done by paid ( and trained? ) staff I would have thought they’d be as good or better than the ones done by volunteers.

September 2, 2019

https://www.duolingo.com/profile/uphilldweller

It seems that the forum is the only place that these tests come to light. I've long noticed a reduction of questions and raised this here yet no one has confirmed this to be a general change. So I assume the reduction in effort required to advance is part of an A/B test.

August 31, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

@uphilldweller -- Looking at your Duome info, you don't (yet) have this reduced/truncated crowns structure.

In the new A/B, each Crown level has just one repeat of each lesson. So if a skill has four lessons in CL0 - you only have to do 4x5 = 20 lessons to take a skill golden.

When Crowns launched it took a lot more repetitions of the CL0 count (12 to be exact). So if a skill has 4 lessons in CL0, it would take 48 lessons to take it gold.

As I type, your gold "Introduction" skills shows 36/36 - so you can't be on the new/truncated Crowns.

Enjoy the extra practice while it's possible!

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

It’s important to take into account that the exercise types vary on each crown level. That is the real problem here.

The crown level 4 which is 100% typing (outside of listening practice) used to have probably 20-40 lessons on average. With 20 exercises (50% typing in the target language) per lesson you could construct 10x(20 to 40) sentences in the target language going through the crown level 4 and it would make tremendous difference in your learning.

Crown levels 0-3 and 5 have a lot of/mostly multi choice exercises that become unchallenging after a while. In order to grow you have to increase the challenge, not lower it.

Then, if Crown level 5 actually offered challenge like Crown level 4, there would be no problem, the lesson count would also be irrelevant as the Crown level 5 lessons are infinite.

September 1, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

@Fuurinkazan -- Below is a thread about a new question type DL appears to be working on. It's "Arrange All The Words".

From what I can tell, Here's how it works: you get a group of target language words and no source language sentence to translate. You might get a corresponding image, giving a hint. The task is to create a sentence out of the words. If you've ever played Jumble - it's very similar. DLs just using words, not letters, for the tiles.

It's part of the reason why I'm wondering if part of the goal is to eliminate "free text" question types. If DL can do so, it has the possibility to reduce tree dev and maintenance created by having to deal with the complex task of covering all possible correct answers. (This has been an issue on the staff developed French tree, where the volunteer contributors have a near-zero role other than answering forum questions. I wouldn't be surprised if this is part of the challenge that DL is facing.)

https://forum.duolingo.com/comment/33657205

September 1, 2019

https://www.duolingo.com/profile/Crystal_Kirscht

I got those for a short time and then they disappeared

September 2, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

@Fuurinkazan - I certainly understand the value of shorting the distance between CL0-5. (Somewhere I could dig up a comment where I broke down how many years it would take to turn the new French A1/2 tree gold in the longer CL0-5. It was ~2.5 years.)

I appear to be in yet another A/B test group where crown level 4 has what I consider to be very little "type in target" because I'm getting a lot of target language "fill in the blank" (etc) - even in CL4.

So much so that it feels like DL is trying to (over the long term) eliminate any/all "free text" question types, even in CL4.

That last sentence needs to be read with a big caveat. It is too soon to extrapolate, and lots of A/B tests never make it into production. I can think of reason why they might choose to do this, but I have no insider info so my thoughts might be off the mark.

What I do know is that my new work process (listed below) is the only way I can learn French on DL at the moment. For whatever tiny A/B group I'm in, there is insufficient challenge/access to content to learn the language.

September 1, 2019

https://www.duolingo.com/profile/ionasky

Except of course it wasn’t like that – you could have 30 attempts at writing the same 30 sentences, 20 in each lesson. If you get them wrong, all well and good then you need to practice, but if you get them right all the time, Or with only the most minor of mistakes and then it is incredibly tedious.

September 2, 2019

https://www.duolingo.com/profile/GingerSquare

When it got tedious, I would test out and move on, but sometimes I needed the extra practice.

September 14, 2019

https://www.duolingo.com/profile/uphilldweller

What I do have (this past month) WRW are 17 questions to answer where there used to be 20. This is what I have posted about and this is what no one has commented on. Hence my suggestion that this was part of an A/B test (another one)?

September 1, 2019

https://www.duolingo.com/profile/sc_ville

I also have 17 on the website, 13 on the app. My level icons other than 5 also move when you hover over them. Only 5 is nondynamic.

September 2, 2019

https://www.duolingo.com/profile/uphilldweller

You're the first person I've seen other than myself report seeing a change from 20 to 17 questions.

September 2, 2019

https://www.duolingo.com/profile/Seattle_Scott

I'm sure this isn't the only change coming as students head back to school and Duo tries to attract more students. As for the easier lessons, if you take a look at www.duome.eu, you can see that a very small percentage of people actually make it to the golden owl, so I'm sure Duo is trying to improve its metrics in that regard.

Unfortunately I haven't seen any changes lately that I consider a plus. I love Duo, but changes have largely been a minus, like the website redesign, the dancing owl congratulating you for 5 and 10 correct answers, the leagues, abandoning levels, animated icons, the naked golden owl, and percentages instead of lesson numbers, staff created trees that lock out longtime mods and contributors and then the staff makes no corrections to the new trees, constant changes in XP and lingots for lessons and xp. Most changes are just an annoyance and don't really matter.

Still Duo has helped me improve my language skills a lot, and I'll continue to use it, although I did cancel my plus subscription. I'll keep using Duo as long as the positive outweigh the negative.

September 2, 2019

https://www.duolingo.com/profile/sc_ville

in a nutshell

September 2, 2019

https://www.duolingo.com/profile/uphilldweller

Good summary.

If we complain about useless or worse changes we are told off as if the only people who care about Duolingo are the ones who lap it all up.

September 2, 2019

https://www.duolingo.com/profile/TENCHIJIN.

Yup, but then again this forum is a tiny minority. There are a lot of users, The majority of all users never even use this site and probably think of duolingo as "that language app with the green bird" and in that group of users is also the group of the silent sufferers that get all these unwelcome changes but remain silent. Then some might enjoy them. Some probably don’t even notice a change and don’t know better if they never benefitted from the abundant typing in the target language magic here on the web version. If I’m correct, about 18 months ago the pre crown level web version was almost 100% typing if you chose to type. There were no multiple choice exercises and the image exercises (real photos back then) were few. I am not quite sure. I have no idea how the web version was before November 2016 when I first joined.

September 2, 2019

https://www.duolingo.com/profile/GingerSquare

I agree 100%. Learning is hard work, and it’s very difficult to truly measure. Teaching is also hard work. This is a ubiquitous problem in education — measuring true learning is hard, so you try to measure something easier (like page views or completion rates) but those things are just proxies. You can optimize the completion rate and page views to the point where you’re not teaching at all. But if you’re an app developer (or a school) and you have to make money, it’s easy to concentrate on the proxy and forget the learning. The only people who will care are those who wanted to actually go to the trouble of learning something. :-(

September 14, 2019

https://www.duolingo.com/profile/Linda7Italian

Fewer lessons? Hmm, will check.

August 31, 2019

https://www.duolingo.com/profile/merkavar

What happens at crown 5?

Is part of this AB test also where the crown 5 skills decay, suggesting you review them? Meaning infinite practice?

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Crown level 5 has infinite practice lessons even without the skill decay. I don’t have the skill decay, I use the web version.

So even when infinite, what’s making the use of the practice lessons inefficient, unchallenging, boring, waste of time, is that they consist of exercise types that are far too easy for someone who is familiar with the skill’s material and has practiced that skill for about 20-30 lessons already.

Just for the record, the easier exercise types are needed and perfect in the very beginning as there is enough challenge in them for the learner when first getting introduced into the material of the skill. It’s not that I am against them, it would be inefficient to have too much difficulty in the beginning, and to expect a complete beginner to write a complex sentence in the target language with little to no practice. I am only pointing out how useless it is to keep the same exercise types and thus lowering the challenge and expect efficient growth from the learner.

Here’s a comparison between crown level 5’s practice lesson and crown level 4 regular lesson exercise types I did recently. In order to make the picture clear (red vs green) I used the skip button for all the exercises that were other than typing in the target language exercise.

Crown level 5 skill practice lesson

From the 20 exercises, 5 were typing in the target language exercises. 5 were typing in English exercises. 10 were multi choice exercises which of 5 were Mark the correct meaning and 5 were Select the missing word exercises.

Crown level 4 skill regular lesson

From 20 exercises, all were typing exercises with 1:1 typing in the target language : typing in English ratio.

Conclusion: Needless to say the supposedly challening Crown Level 5 is not really challenging at all if you’ve went through the same material on Crown level 4 with 100% typing. Naturally the challenge should become higher, not stay the same, and especially not become lower.

So what the problem here is, is that the challenge is not increased as you get better and proceed, but taken away from you.

For that reason, I can’t effectively use Duolingo unless they increase the challenge with the exercise type ratio on crown level 5 practice to at least to the difficulty of crown level 4 (100% typing) by default and if you fail them miserably, then give the easier exercise types to build you back up again. Preferably, give us an option to enter a regular lesson or a practice lesson in a 100% typing in the target language mode, or at least 100% typing like crown level 4 regular lessons, or change the crown level 5 difficulty to even higher than crown level 4 (even more typing in the target language vs UI language, e.g. 65:35 vs 50:50).

September 1, 2019

https://www.duolingo.com/profile/Mereade

Thank you for this overview. It really makes the problem really clear and I agree 100%.

I'd say that in this situation, the most logical way to use Duolingo is to finish it and go away as fast as possible, because there is nothing valuable to stick around for. If Level 5 was at least like level 4 (in the exercise composition that you've demonstrated), it would make more sense for people to come here and review after having finished and moved on to other resources.

September 1, 2019

https://www.duolingo.com/profile/garpike

Bringing back skill decay would be a marvellous thing, although it would be a great pity if DL compensates for this by dumbing-down the difficulty of higher crown levels.

September 1, 2019

https://www.duolingo.com/profile/merkavar

Is less lessons actually dumbing it down?

My understanding was crown 5 was so many lessons because there was no skill decay.

So if skill decay is added then shortened lesson counts make sense.

September 1, 2019

https://www.duolingo.com/profile/garpike

The OP also implies that questions have become easier. As for 'infinite practice', one can already do that now in gold skills; one simply doesn't know when the optimum time to do so is as there's no decay.

I always thought the number of lessons in crown levels was simply an arbitrary value to make them progressively more difficult to achieve, and thus encourage more exposure and practice at a comparatively-early stage of learning the material. There's nothing wrong with this, in itself, with or without decay.
Spaced repetition specifically targets material on the basis of when the last exposure occurred, and periodically increases this according to a formula. These two features do not do the same thing.

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Specifically, the ratio of easier exercise types (image, multi choice, match the pairs, fill in the blank and even typing in the UI language) vs typing in the target language is INEFFICIENT on Crown level 5, it’s similar to that of Crown level 0-3. Only on the crown level 4 you get challenging exercises with 50% type in the target language.

And now that there are fewer lessons available on each crown level (80% reduction) the crown level 4 (100% typing exercises outside listening exercises) which is the most challenging level, offers now very few lessons that you cannot repeat as you will have to progress to Crown level 5 eventually.

Solutions

  • Additional Lesson mode where you can force 100% typing (ladder trees), typing in the UI language (reverse trees), or typing in the target language (regular trees)

  • Make Crown level 5 more or at least as challenging as crown level 4 by making it 100% typing and better yet, have it respond to the user’s success or lack of success: failing the questions knocks down the challenge and make it feed you the easy exercise types to build you back up, and on the contrary, increase the amount of typing in the target language exercises if you successfully get them right time and time again. Plus, have the SRS skill decay so you can efficiently return to those sentences that you need to drill. This way the amount of lessons become irrelevant as the crown level 5 (infinite lessons) will be challenging.

Really don’t understand why they can’t create a system that will (within each course) respond to the user’s success and add the needed challenge. It’s shouldn’t be rocket science. Or if it is, make it fixed/static (don’t know a proper English word to use here), so that on a higher crown level, you will have more challenge and after x amount of fails you get knocked to a lower crown level for example. Then the fear of too much challenge on crown level 5 or 10 (if there were 10 of them that is) would not scare away the beginners (beginner in the sense that the user has not mastered the material within that specific skill) because there would be no beginners on that crown level in the first place.

September 1, 2019

https://www.duolingo.com/profile/Mereade

Well, I think your point about a specific just typing mode is the best idea. If I could just go to the settings and switch to typing, and from source to target language only, I would be happy. And other learners would not choose that (I can definitely imagine I wouldn't want that, if I was learning Hebrew or Mandarin) and would be happy too.

It is just sad that Duo doesn't see this digital platform as a wonderful place for individualised learning. Instead, they are try to click measure their way to some dream of a universal one size fits all solution. Which is nonsense.

Such a typing only mode (a choice!) might solve the majority of the common complaints concerning the learning mechanisms.

If I could just type and leave out the exercises wasting my time, I would be all for the current shortening of the skills, it would actually be a very good idea in such a case.

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Spot on!

September 1, 2019

https://www.duolingo.com/profile/sc_ville

I really like this idea. Conceptually, they're already doing it with leagues. Shouldn't be so very hard to apply that same logic to the actual lessons.

September 2, 2019

https://www.duolingo.com/profile/gwen756874

The gold skills practice is very dumbed down though. I tried it when they went to 5 lessons/level at all levels recently. But it’s just choose the word from word bank. Which is garbage not practice

September 15, 2019

https://www.duolingo.com/profile/W-Ruggles-Wolfe

I'm hoping that is where they are going (shortened crowns, bring back SRS). That what we are seeing are the interim steps necessary to evolve the system.

I'm less optimistic about the question type changes. I'm trying to keep an open mind about that one.

September 1, 2019

https://www.duolingo.com/profile/testmoogle

A "new" one? When did it begin happening? I've been somewhat keeping tabs on A/B tests recently but I haven't noticed the A/B test you've mentioned.

Currently there are at least 132 active A/B tests running. Very few of these currently active A/B tests look like they have any potential to cause the effects that you've mentioned.

I checked out a few of the higher potential looking A/B tests to see if being in their experiment groups cause it, however none of these did for me. So I'm thinking it might be one of the platform specific A/B tests? I've only been trying it on web.

Which platform are you exeriencing this on? (Android app? iOS app? Web?)

Also, which course did you exerience it in?

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Web (it’s what I use 99% time) and iOS, all courses affected.

September 1, 2019

https://www.duolingo.com/profile/testmoogle

Thanks for providing the specifics. Now I've got a much better understanding of the situation. ^^

Also, I've now seen how on your Duome page you have, for example, a crown level 4 skill ("Cores") which says it contains only one session. This shouldn't be possible as crown level 4 is meant to repeat each lesson five times, so the number of sessions in crown level 4 should always be a multiple of five.

What's strange is that I've tested all nine of my accounts and not a single one of them is currently under the effects of this mystery A/B test experiment, as can be seen on their Duome progress pages:
1 2 3 4 5 6 7 8 9

It might be worth never levelling up any more skills to crown level 5. Once you reach crown level 4, simply stop and use Skill Practice early instead.

For example:

The start next lesson button for your crown level 4 "Cores" skill would take you to this page:
https://www.duolingo.com/skill/es/Colors/1

You could delete the lesson number ("1") at the end of the page address and replace it with the word "practice". This way allows you to do Skill Practice at any crown level, rather than having to wait until crown level 5:
https://www.duolingo.com/skill/es/Colors/practice

I haven't really been able to test whether the crown level 4 Skill Practice works as good as the crown level 4 lessons. It might be worth a shot though, unless you've tried this already. ^^

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

No problem, and yes it is strange that there is only 1 lesson for colors/cores on crown level 4. It used to have 3-4 at level 0-1 if not more, if I remember correctly,

Also, did some further testing:

Crown level 4 Skill practice

  • Mark the correct meaning x7
  • Write in the UI languge/English x7
  • Write in the target language x6

Crown level 4 Skill timed practice

  • Mark the correct meaning x7
  • Write in the UI languge/English x7
  • Write in the target language x6

Crown level 3 Skill practice

  • Mark the correct meaning x6
  • Write in the UI languge/English x7
  • Write in the target language x7 img

Crown level 3 Skill timed practice

  • Mark the correct meaning x7
  • Write in the UI languge/English x7
  • Write in the target language x6 img

Conclusion:

  • Skill Practice Lesson on Crown level 3 is more challenging than on Crown level 4. Again that weird logic, lowering the challenge on a higher crown level.

  • Skill timed practice lessons on Crown level 3 and 4 are identical to the Skill practice lesson on Crown level 4.

September 2, 2019

https://www.duolingo.com/profile/David666253

I am very discouraged by this change. I felt like I was making good progress but now I've lost motivation to continue.

September 2, 2019

https://www.duolingo.com/profile/PatchyPatchy

I am part of the A/B test. The lessons changing is terrible, but I like the other changes to the app.

August 31, 2019

https://www.duolingo.com/profile/JudithAnne119783

What's an A?B test I'm scared LOL.

September 1, 2019

https://www.duolingo.com/profile/garpike

'AB' was the ancient Egyptian conception of the soul, weighed after death on scales against the feather of righteousness by the goddess Maat, and, if found wanting, cast aside to be devoured by the monstrous serpent Apep.
In 'AB' tests here, our accounts are weighed before death against the feather of profitability, and Duolingo generally takes on the role of Apep, regardless...

September 1, 2019

https://www.duolingo.com/profile/schokolade

In an A/B test, you have two variants and give half of the people one variant (version A), half of them the other (version B), and then see what happens.

For example, imagine you have an online shop. The "add to basket" button is red. You have a theory that if you make it green, more people will buy things. So you do an A/B test where half of the people get the red button and half get the green button, and then count how many sales you make for each group. If you get significantly more sales from the green button, you would conclude that a green button is better and use green for everyone.

September 1, 2019

https://www.duolingo.com/profile/TechNoir_

Not enjoying the reduced lessons! I'm flying through subject matter and don't feel like I'm learning like I did before. Really enjoyed practicing with words up to 20 lessons at a time. Completing all levels isn't an achievement anymore. Feels like the app is pacifying impatient users.

September 2, 2019

https://www.duolingo.com/profile/betsys2003

I have the fewer lessons to level up part of the test, but haven't noticed any difference in the type of questions I'm getting. I wonder if that's a separate A/B test?

I don't really mind the fewer lessons bit. I wouldn't have gone as far as they did, but having SO MANY like they did before (I had at least one that was 32 lessons from 4 to 5 - and it was an early lesson so it was super repetitive) was probably making many people discouraged and/or test out instead of doing them. With 4-5, there's less incentive to test out so on average people might actually be doing more lessons.

The fact that they're re-introducing decay, and that you can of course continue to practice skills if you feel you don't know them yet at gold, adds to this. If you feel you haven't mastered it by the time you make it gold, use the practice function.

September 1, 2019

https://www.duolingo.com/profile/betsys2003

To check, I just did a lesson in the "from crown 4 to 5" section (last group). In English-to-Spanish, in the second grouping if that matters (Phrases 2).

There were: 3 "speak what you see"

4 "type what you hear"

3 "spanish-to-english"

7 "english-to-spanish"

0 picture or multiple choice

So out of 17 lessons, 7 (over 40%) are "translate into Spanish" which most people agree is the most difficult. That seems like a pretty good amount to me. More of that would take away from the other three types which have their own benefits. I know the "speak" ones seem easier but I think it's really important to practice saying the words. The dictation ones test whether you are hearing things correctly and understand spelling and sounds (they're not super hard in Spanish because of the pronunciation of that language, but in other languages are often very challenging). And even "translate into your known language" is important to do sometimes, because that's what you're doing quickly when you listen to others talk.

September 1, 2019

https://www.duolingo.com/profile/Mereade

Thanks for this example, it is very interesting!

While I'd agree that dictation and translations to English have their uses (the speaking exercise is broken, you are practicing just as well or better by just repeating after the audio, but these can fortunately be turned off), I'd leave them at the lower levels. Only 40% of the questions in your mix were the ones I want from Duolingo. That is rather sad.

September 1, 2019

https://www.duolingo.com/profile/betsys2003

I thought the speaking exercises were "broken" until I tried some in the reverse tree (so the speaking exercises were English) and I got every word perfectly every time. I have also noticed that as I've gotten more comfortable with Spanish, I get more of the speaking correct.

Could you get the same result from just repeating other questions? Sure. But most people won't do that. They need a specific instruction to "say this now." Duo should not be designed around the top 5% of users who will go above and beyond what they are asked to do, it should be designed for the average user who will not ever say a word out loud unless directed to do so by Duo.

September 1, 2019

https://www.duolingo.com/profile/Mereade

Well, I had to fake a typical English native's mistakes to get some answers accepted. And I know a learner, who got his cough accepted. I don't doubt some answer get judged correctly, but I don't think it is a reliable tool.

Yes, you will get very good results, if you just repeat after the audio and try to be as similar to it as possible. It works.

Most people won't do that, true, but: 1.Perhaps there should be some study tips easily available on the website, including basic advice like this. It is a logical thing to do, but most newbies may not think of it, true. 2.It is actually the individual learner's responsibility. And I don't think introduction of low quality exercise for everyone is the best solution to some learners simply not thinking of the better solution. 3.If a learner is really that passive, I wonder why are they even here. Are they being told at home "do two lessons of Duolingo today?" :-D I guess most are not. So, perhaps being told this basic tip once or twice would help. If it doesn't, then it's the learner's fault. You know, if someone needs to be told what to do all the time, perhaps they should just get a teacher and give up on independent learning.

September 1, 2019

https://www.duolingo.com/profile/stewacide

I really dislike how dumbed-down the app version is. Doesn't teach anything so I end up using the website version 100% of the time even though it'd be convenient to use the app if it had the same content. PLEASE don't make the website like that!

September 1, 2019

https://www.duolingo.com/profile/uphilldweller

It took a year or more though pop up owls finally made it webview - so fingers crossed.

September 1, 2019

https://www.duolingo.com/profile/Epicity

How does Duolingo measure A/B test metrics on users who would use this service no matter what since they want to learn?

How exactly do we let duolingo we don't like the new features but keep using it? A/B testing kinda fails here.

September 1, 2019

https://www.duolingo.com/profile/corgi.

This would be an absolute tragedy. The Duolingo lessons are fine the way they are. If anything, make them harder with more typing questions! Level 4 is the most substantial lesson that really helps nail the knowledge into your brain, they can't get rid of it! I hope that test flops horribly, or that the change at least doesn't effect older accounts. Why does Duolingo insist on doing these horrible revisions nobody asked for? Either leave it alone, or listen to what the commenters are requesting.

September 4, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Why This Founder Says the Worst Advice He Ever Got Was to Listen to His Users

https://www.entrepreneur.com/article/290664

By Luis Von Ahn, the CEO of Duolingo:

What's the worst piece of advice you ever got? I don’t know if it's bad universally, or if it was just bad for me. When you’re building a product, typical advice is listen to what your users are saying. I have found that is terrible advice for Duolingo. The spirit of the advice is good, but a lot of times, there is a problem that arises. If the feedback channels are such that that only a few users get to talk to you, you are only hearing the loudest ones, not all of them, and that is where it breaks down.

We have a forum on our website where people can talk about language, but they often ask about features. I have found that listening to people in the forums is a terrible idea. For example, we redesigned the website a few years ago,and people in the forum were saying how terrible it was. Hundreds of posts about how this was the worst decision we had ever made. All the while we were looking at the metrics for the new website, and they were significantly better.

If you are just listening to the people who reach out to you, than that is a biased sample of people who are a loud minority. Of course that is not always the case, but unless you know what you are doing, you should watch out for that bias.

I understand what he’s saying, and dumbing duolingo down will give better metrics as the challenge is low, but it comes at the cost that the tool is not efficient for that long as the long time users of a specific course are fed the same exercises as the beginners of that same course. Or someone with 20-30 lessons on his belt in a specific skill is given the same lessons as someone with 0 lessons on his belt.

I was doing over 400+ xp per day for about 2-3 weeks and before it I was doing 200-300 xp per day for about 2 weeks or more. I was trying to get my trees to crown level 4 where the typing exercises are. Now that the crown level 4 lessons are reduced by 80% I do about 20 xp per day with this inefficient set up.

The metrics of my account must be really good for them.

Really wish they would do something about the crown level 5. Enable additional practice lesson mode with 100% typing with 3 options: 50:50, 0:100, 100:0 typing ratio between the user interface language and the target language, available at all times. Or let us check/uncheck the exercise types from the options.

I want to do more lessons but I don’t want to see those pictures, multiple choice, fill in the blank, match the pairs exercises. I’ve become allergic to them. They are fine on the lower levels and I need them there but not on Crown level 5 or after 20-30 lessons within a specific skill anymore.

September 4, 2019

https://www.duolingo.com/profile/thomasleeevans

At age 66 and retired I have found that the more repetition the better I remember the material over time. I generally have three skills in progress and rotate through them until complete. As I am attempting to learn a language, not play a game. I find the change has reduced my retention and requires me to do much more work outside of Duolingo.

One can only hope the staff will moderate the change by increasing the difficulty and lesson content.

September 5, 2019

https://www.duolingo.com/profile/George185298

I am also really disappointed with this update. Apparently this is for locking in new people or give motivation to people who just want to play a "game" and would like to use duolingo as such. For people who are interested in actually learning a language and improve in it, it was the worst they could do.. to be honest i am not learning it as fluently as before and it is not as appealing or motivating to me to earn points or whatever. The true reward comes when you can understand and use a language. Why oh why, when you already have something perfect you try to mess it up by trying to make it better... This was the worst decision/update in the history of duolingo! If duolingo doesn't bring back the normal quantity of lessons which is essential for a language to actually settle in, many motivated people will stop using it...Period...

September 14, 2019

https://www.duolingo.com/profile/GingerSquare

I will be one of those people. If they don’t fix this, I will ride out my one year subscription and then cancel.

I’m definitely not going to work any more skills to gold until this is fixed.

Also left a one star review in the App Store. I hope everyone who hates this change will do the same.

September 14, 2019

https://www.duolingo.com/profile/TylerMuell10

I am relatively new to Duo, having achieved a level 5 in just one topic, which I think involved 38 "tests", so I do not have the experience of most people here, but I did think that for my purposes, it was too long. What I want from Duolingo is to get word and word patterns engraved into my brain. I have read elsewhere that a newbie language learner needs to use a foreign word at least 30 times before this happens (thats on average, with me its more), In those 38 tests of around 12 questions, I must have met some words at least 100 times - which I think was overkill, even for me. For this reason, I would not mind a reduction of the work in a module. Having said that, even in the 40 days or so I have been using Duo I have noticed changes to the course structure. It must be unsettling for the old hands.

September 1, 2019

https://www.duolingo.com/profile/Mereade

Yes, you are absolutely right that some skills were really an overkill. Especially the really basic ones. But the problem is not shortening the skills, it is removing the useful exercises and leaving the worse ones.

Yes, you need to use the word many times. But the best way is active recall. Multiple choice is not too much of active recall, translating to your native language is not much of active recall. Typing the translation to the target language is the best for engraving new words and patterns into your brain, as you said. That's what has been great about Duolingo.

Unfortunately, this is not about changes being "unsettling for the old hands". It's about the "old hands" observing a series of changes to the worse.

This is just a test, perhaps it will really be a failure. But who knows. It is unpredictable. Perhaps it would be the best, if Duo got finally divided in two products. One for people who want a gamified learning tool, and the other for people who want just a game with a language theme.

September 1, 2019

https://www.duolingo.com/profile/Fuurinkazan.

Yes, you are absolutely right that some skills were really an overkill. Especially the really basic ones. But the problem is not shortening the skills, it is removing the useful exercises and leaving the worse ones.

Exactly! If the crown level 5 practice lessons were challenging (100% typing outside of listening exercises), there would be no problem.

The strategy before this ab test was to avoid crown level 5 like a plague. The huge overkill amount of lessons on Crown level 4 was the solution for it. You could do 4-7 lessons and go through all the material in a few days, then come back to that skill a week or a month later and repeat the same material, and then come back again later. You could be doing this every day going through many different skills’ crown level 4 lessons little by little and only reach the crown level 5 after a long period of time.

Preferably Crown level 5 would be challenging with SRS skill decay and we would have additional lesson modes (100% typing in the target language, or UI language, or 50:50 target:UI). Then this problem would be non-existent.

September 1, 2019

https://www.duolingo.com/profile/ISpeakAlien

So far, I have not had to translate one single sentence from English into Latin on the Latin course. I am almost to the second checkpoint.

September 1, 2019

https://www.duolingo.com/profile/Sharon275517

You'll get English-into-Latin translations at higher crown levels.

September 13, 2019

https://www.duolingo.com/profile/Algarve11

Oh God. Why do they have to spoil everything?

September 1, 2019

https://www.duolingo.com/profile/YarkoRab

That's bad news, isn't it? I'd rather they made the course harder ...

September 3, 2019

https://www.duolingo.com/profile/Basil598271

I agree 100% with comments. This change makes duolingo almost worthless. I need a stronger challenge and more lessons at level 5 not less. This test needs reversed immediately. Very unhappy

September 13, 2019

https://www.duolingo.com/profile/nye.simmon

This is a terrible update. It is easier with less learning. Please change it back! We are here to learn not just play games. Thank you!

September 14, 2019

https://www.duolingo.com/profile/CGHC30

I, too, am less motivated to move forward, knowing I have not yet mastered the levels I was suddenly promoted on. I NEED the practice that I was enjoying on each of the levels in a skill. I had been impressed with the former progression of skills but the current situation, not so. I have tried to practice the skills that I missed but since the repetition for typing the words in level 4 has disappeared, I am unable to spell the vocabulary needed and can not reach my practice goals of 18+, and in fact, can only get 9 - 10 correct in a practice session. Frustrating.

September 14, 2019

https://www.duolingo.com/profile/Rosalie_L

NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

I hope so much that this doesn't affect the French course.

August 31, 2019

https://www.duolingo.com/profile/tyouyoo

The lessons in French are less than before

September 1, 2019

https://www.duolingo.com/profile/Rosalie_L

Mine aren't, so I guess I'm not on that side of the AB test.... For now.

September 1, 2019

https://www.duolingo.com/profile/Mary432092

Ugh, I just got this update. Thankfully, the global practice button helps. That focuses mainly on the current lessons I'm working on and throws in a bit here and there from older lessons.

September 1, 2019

https://www.duolingo.com/profile/phil464754

Is participating in these A/B tests compulsory? Has anyone been successful in being removed from these tests? How do they pick the guinea pigs/victims for the tests? Is there a recent post from DL explaining what they are testing (if so please would you be so kind to share the link) Thanks in advance

September 1, 2019

https://www.duolingo.com/profile/Mereade

Yes, they are compulsory.You become a guinea pig by starting an account. You don't stop being a guiney pig by paying for Plus (that is the weird part!).

No, there is usually no post, only if too many people complain. Some changes get announced, but rarely with any thought for possible inconveniences and the post is much more marketing focused than information focused.

Yes, there is a way to avoid some tests. Some users have multiple accounts and just pick the one that fits their preferences. For example, when a new tree is being rolled out to some newbies, there are people who make new accounts until they finally get the tree they prefer :-)

September 1, 2019

https://www.duolingo.com/profile/DavidSwarl

This really is diminishing Duolingo's value. I don't want to keep track of skills I am not so good at and the amount of lessons is way too low to master them before achieving level 5.

September 2, 2019

https://www.duolingo.com/profile/lawriegloria

It is so sad, the new changes have ruined the enjoyment of learning with Duolingo for me especially the heath changes. I look enviously at my husband’s version with no health, still all the skills removed from mine and the extra lessons for each level. How can I get his version?

September 3, 2019

https://www.duolingo.com/profile/MatiDomo

I'm also affected by that. I pay for the Plus and still I'm a guinea pig.

September 3, 2019

https://www.duolingo.com/profile/gwen756874

It’s like they are rewarding the idiots who use this as a game to climb the tree quickly. Do they want to be a game or a teaching app? That is the question. I was actually learning French. I skype tutor with someone once a week and he’s been amazed at my progress with speaking lately. I’ve been learning French for one year. The new tree that came out forced me to go back and review. The lessons introduced a perfect number of words per lesson and connected them better to the usage topic. I would go through a “castle” at one level, then come back and progress through the next level. And so on. By the final level I would be able to form my sentences in French well and talk to text them and be understood by my phone. That’s actually learning something. This A/B test is absolute garbage. Hear me Duo!!??

September 15, 2019

https://www.duolingo.com/profile/Jonathan146948

Anyone else seeing now on iOS that with each crown level there is higher XP to be gained?

September 1, 2019

https://www.duolingo.com/profile/betsys2003

Yeah, I've got that. I like it. It's a lot harder to do a crown 4 lesson than 1 so now this matches a bit.

September 1, 2019

https://www.duolingo.com/profile/KerriVande1

Yup, I've noticed this. Not sure how I feel about it since I can get my daily goal in one lesson sometimes.

September 1, 2019

https://www.duolingo.com/profile/Doug717439

I believe you’ve left at least one part of this AB test—“broken” units. Some units appear cracked and you have to do a lesson to restore them.

I think this counteracts what you’re calling the dumbing down of the software.

Also, I’m happy to have fewer exercises to level up to 5. They seemed repetitive, forcing me to memorize specific sentences rather than grasping grammatical concepts. Often I’ve reached 4 and then test up to 5.

My POV: 400 some continuous days on DL, learning Dutch. Well over 50 yo.

September 1, 2019

https://www.duolingo.com/profile/KarenSharbo

What is the AB test?

September 2, 2019

https://www.duolingo.com/profile/fraisdebois

fewer lessons

August 31, 2019

https://www.duolingo.com/profile/betsys2003

I'm pretty sure you were correcting the use of "less lessons" to the correct "fewer lessons" but with just two words it's not clear that's what you're doing. Try using more words next time.

September 1, 2019

https://www.duolingo.com/profile/Carmen807459

Speaking of A/B tests, my French tree was re-upgraded. I guess it's "tree 13", but I now have fewer crowns to complete the tree (712 crowns).

I'm fine with a more condensed tree, but I wish there was more information given, like whether or not skills/vocab has been lost or which topics they decided to consolidate.

It seems that W-Ruggles Wolfe suffers the same fate as me.

September 1, 2019

https://www.duolingo.com/profile/Wayne____

I haven't seen it and it doesn't seem like a positive change. However, Duolingo is first and foremost a business. They make more money with more users who stick with it longer. The typical user probably starts off and drops out within a few days or weeks. If they can double the time that those users stay, they'd make more money even if they lose all existing dedicated students. I'm not suggesting that that's the planー but they do have a lot of things to consider, including getting more users and retaining them.

In my view, a better approach would be to offer multiple tracks, leaving older ones for people who still want to use them. The risk they run from alienating existing users is getting poor reviews elsewhere that discourage new users.

September 2, 2019

https://www.duolingo.com/profile/SamohtSMA

Before this revision, I worked through all the lesson on levels 1 through 4. Those 28 lessons were enough for me to pick up the new vocabulary, practice the new grammar twists and work on listening skills. At that point I switched from the app to the website and tested out of level 5. I paced myself at 5 crowns in 4 days ( 2, 1, 0, 2 per day). The 20 lessons at level 5 were just drudgery so I tested out.

This new lesson series of 4 lessons per level cuts out the drudgery, but I find I make more mistakes on the upper levels. I’d prefer a few more lessons at level 3 through 5, maybe 8 instead of 4.

September 3, 2019

https://www.duolingo.com/profile/tonyph17h

I have reached Checkpoint 4 on 2 courses recently and am having to pass a Checkpoint Challenge before the next skills become available. I this a new A/B Test?

September 3, 2019

https://www.duolingo.com/profile/RyanGomez192917

I'm attempting to grade my students' performance on Duolingo (their homework) but for some reason some students have received credit for only completing one lesson of an assignment, while others have completed three or more. This makes Duolingo useless when it comes to grading, and totally unfair to the students who are basically being asked to do more to receive the same amount of credit for the same assignment.

Someone kindly let me know about this A/B testing, and that this problem I'm experiencing is likely a result of one group of students having the easier workload, and the others not. This is still absolutely ludicrous. I don't understand why they'd even offer the programme as a school homework option knowing that they're doing this. If they're going to ask students to do different amounts of work for essentially the same credit, Duolingo has no business being a primary homework option or being marketed to university language programmes as such. This latest outrage comes on top of the generally convoluted setup and user processes with which this programme has already presented us. And it seems like one problem or issue after another arises every couple of weeks.

My particular problems aside, the concept of dumbing down the software is frankly moronic since it was already very easy to being with. It was a very poor decision on their part, and frankly, I hope this backfires for them big time.

Outrageous stuff.

September 16, 2019
Learn a language in just 5 minutes a day. For free.