https://www.duolingo.com/JustinSensei

Experiment: Add to Tinycards from Hints in Duolingo

Hello! The Tinycards team is experimenting with a new feature that lets you add cards to a special flashcard deck in Tinycards whenever you view a hint in the language you're learning in Duolingo. The goal is to make it easier to practice the specific words that you want to practice the most.

We’re starting the experiment with a very small number of users on web. If it’s successful, we’ll bring more and more web users into the experiment, and if that’s successful we’ll add the feature to Android and IOS at a later date. I’ll use this post to keep you up to date as we work. And as always, feedback is very, very welcome!

UPDATE: The Tinycards team has decided to shut down this experiment. While it has been popular among a subset of users, on the whole it has had a neutral to slightly negative impact on the learning experience in Duolingo. Going forward, we'll be looking into some other ways we can make it easier to practice specific words from Duolingo, ideally without requiring the use of a separate app.

Note that decks created via Duolingo hints will be available indefinitely, but as we shut down the experiment, you will no longer be able to add new hints to any existing hints decks.

August 10, 2018

72 Comments


https://www.duolingo.com/Kuraikari

That sounds great! Are there going to be unit tests or just the whole feature? What about the listening exercises?

I'm happy to see that Tinycards is getting more integrated into Duolingo :)

August 10, 2018

https://www.duolingo.com/chris627090

That’s a real shame, i hadn’t used tiny cards before because it was too difficult to find specific vocabulary, but adding from duo, with the example sentence was brilliant... hope you change your minds

September 13, 2018

https://www.duolingo.com/RogueCheddar

Having a dictionary with all the words the user has learned would be great.

August 11, 2018

https://www.duolingo.com/Medard

It is available but just for Spanish, German and French course.

August 11, 2018

https://www.duolingo.com/dogomolo

Why?

August 11, 2018

https://www.duolingo.com/Humamslayer12

why what ?

November 23, 2018

https://www.duolingo.com/jolynnedougherty

Nice work Justin and team! Looking forward to trying this :)

August 12, 2018

https://www.duolingo.com/Thomas.Heiss

@JustinSensei

Hi,

is your team aware that ALL auto-generated TinyCard decks / cards either from the normal lesson or story generator have no working audio?

Why is that?

When I create a manual TinyCards deck I can choose the audio (TTS?) for the front and the backside in the deck settings.

The cards of the "My DuoLingo PT words" show the audio button, but it is not working.

All generated cards from the story decks of the user "Duolingo stories" within the "Duolingo PT stories" deck even do not contain this audio button in the top right corner.

The symbol is missing, so obviously no audio mapping has been chosen by the generator?


Question:
Is it possible to do a CSV export to Memrise or AnkiSRS from TinyCards?

We (the community) have a very powerful Tampermonkey user script support for the Memrise web portal and within AnkiSRS there is the "Overdueness" support when you have to review out of the standard spaced repetition (SR) intervals because of longer breaks.

Sorry, I can not find anymore the other TinyCards announcement threads for the story generator so I could post my problem about the audio.

Thanks for taking your time.

I greatly appreciate your added TS feature to mark specific words with the full sentence as an example in the normal Duolingo review practice session.

With best regards

August 31, 2018

https://www.duolingo.com/JustinSensei

Hi, Thomas! For the moment, it's expected that you'd see no audio files for the Stories decks - there are still a couple wrinkles we're ironing out before we turn text-to-speech on for those decks. (But it should be available soon!)

As for decks created via Duolingo hints, those should have audio enabled, and for what it's worth I'm seeing audio in my own German hints deck. If you could tell me a little more about which language you're studying and which platform you're using for Tinycards (web, Android, iOS) I could take a closer look for you.

Regarding CSV exports: would you be open to starting a new thread on that topic? I'm guessing others would be interested in joining the discussion who wouldn't be likely to find your question in this thread. Thanks!

August 31, 2018

https://www.duolingo.com/JustinSensei

Hello, again! Just letting you know that audio should now be available on decks created from Duolingo Stories. Enjoy!

September 4, 2018

https://www.duolingo.com/mahu11

I only started using them but I really like the ability to store specific words that I need to remember.

September 14, 2018

https://www.duolingo.com/AmyBygler

I have already commented, but would like to add to that. Why doesn't Duolingo listen to what people are saying, rather than citing data and deciding on what is good for others? If people are telling you it works for them, that they are more proficient and have better recall, it certainly would seem to be helping. For those that don't find it useful or that it impedes their learning, they can simply ignore that feature. Duolingo is seriously underestimating the intelligence and decision making capabilities of their customer base. These are people that are intelligent enough to learn and absorb non native languages. These people are surely able to make use of, or dismiss features according to their style of learning. Is it not possible that the negative impact data that has been collected is reflective only of the customer base it does not work for, and conversely that there is still a base that is does work for, and that these people are capable of using/not using said feature as it suits them. Removing it ignores those it does help and completely takes away their choice in the matter. People have different ways of learning. Removing avenues to those different learning pathways is an antiquated approach, a mistake made for many many years in the school systems, leading for example, to seriously poor learning outcomes for those students who did not fit into the "prescribed method" of learning. You should rethink this decision Duolingo.

September 14, 2018

https://www.duolingo.com/piguy3

B/c the people that are showing up to say things are a thoroughly unrepresentative sample. Those that got confused or disengaged don't show up to complain. That's why data beats commentary. There will always be response bias in commentary.

There's this annoying thing called the law of unintended consequences. Obviously the TinyCards team didn't go to the trouble to design and implement this experiment thinking it would have an average (if statistically insignificantly, if I read between the lines of JustinSensei's update) negative impact on people's learning experience. That fact alone shows you it's unintended consequences in action. People find themselves distracted, confused, disengaged for an untold plethora of reasons. The ability to ignore the whatever leads to this doesn't in and of itself allay this reality.

Duolingo runs hundreds of experiments. Most of them probably help some people while hurting others, leaving the overall average not-too-changed. Sensibly enough, they target their efforts on the changes that have a more positive impact than that, just as they have here.

September 14, 2018

https://www.duolingo.com/Bethany281730

That sounds awesome! I would love to see that feature added to Duolingo. Would it be a different deck for each language, or just one with words from each language that you're studying in it? I personally think that a separate deck for each language would be much easier.

August 11, 2018

https://www.duolingo.com/JustinSensei

Thanks! You would get one deck per language you're studying in Duolingo. So if you add hints to Tinycards from German, and you add hints to Tinycards from Spanish, you'll end up with one German deck and one Spanish deck.

August 13, 2018

https://www.duolingo.com/Bethany281730

Okay, thanks for explaining.

August 16, 2018

https://www.duolingo.com/Melyndi

oh my so awesome! I just discovered this today. I hope I'm a good test subject. I don't use hints that much any more... I guess I might have to start using them again

August 13, 2018

https://www.duolingo.com/Gail667569

I'm tired of writing out index cards to please continue with your experiment, please!

August 16, 2018

https://www.duolingo.com/Humamslayer12

it was cancelled...

November 23, 2018

https://www.duolingo.com/qomanaut

They found out that some people spend less time with that feature and therefore they cancelled it for everyone.

November 23, 2018

https://www.duolingo.com/KarinSumbl

This is a different tiny card question: How do you put accents on your cards or in your answers when you play the cards?

August 30, 2018

https://www.duolingo.com/maudape

Dear Duolingo team, First of all, thank you for the great job of Duolingo. It is a real help for all of us wanting to learn and improve our language skills. Thank you so much!!

Regarding the experiment of the integration with Tinycards, I share the frustration with other users, but also I understand your points. I was using Tinycards and got interested in Tinycards for the only reason of the words integration with Duolingo. It will be awesome if the team manage to include this feature in Duolingo itself later. We will wait for it!!

Thank you very much for your hard work!

Best regards,

Mauricio

September 15, 2018

https://www.duolingo.com/nrDHqU

I am disappointed to see this feature removed. Often, the cards shown are not what I need help recalling.

Right now, my practice sessions are not really referencing much of what I am learning in my newly formatted skills lessons. Some of these practice sessions test the progress I made before the learning break I took for almost a year; this is great. However, some things I have never learned. Still, I use the prompts to help me learn the new things, but since they are part of the practice and not a skills lesson (so they appear intermittently rather than being drilled and progressed) I really liked adding them to Tiny Cards so I could drill them and not be stumped the next time a new or seldom used word or phrase is presented.

Please bring this back along with the grammar lessons that were at least available on the desktop version. Some lessons need a clear explanation rather than checking the feedback and reading through all the other frustrated and confused reader's comment to find 2 or three competing responses. I often read the community; it's great. Sometimes a quick explanation of the rule with simple examples makes for faster and longer-lasting learning.

Thank you!

September 17, 2018

https://www.duolingo.com/Jiiwooo

Ahmmm... How can I find the tinycards for japanese can someone link it please?

August 10, 2018

https://www.duolingo.com/JustinSensei

Right now, we have two official Japanese decks in Tinycards.

Katakana: https://tiny.cards/decks/29B5nanR/writing-japanese-katakana

Hiragana: https://tiny.cards/decks/e1fhs/writing-japanese-hiragana

August 13, 2018

https://www.duolingo.com/Ginasem

That sounds really great.

August 11, 2018

https://www.duolingo.com/illeionathan

This sounds fantastic! I was really bummed out when the Spanish Booster Deck went away and am glad that it's returning in some form, and this sounds like the perfect way to do it. Looking forward to it!

August 11, 2018

https://www.duolingo.com/piguy3

That sounds like a wonderful feature!

August 11, 2018

https://www.duolingo.com/jegviltaledansk

Hi @JustinSensei. Does Duo plan to launch the official Tinycards deck for Danish in near future?

August 11, 2018

https://www.duolingo.com/JustinSensei

Hello! Part of what we're measuring in this experiment is whether users will need official course collections and the personal decks they've created from hints in Duolingo. If the experiment is successful, we probably wouldn't add more official course collections in Tinycards - instead, we'd let users curate their own decks to use alongside Duolingo. (And remember, the decks created from Duolingo hints will earn XP!)

August 13, 2018

https://www.duolingo.com/tonyph17

Hi@JustinSensei As far as I am concerned both features are needed. Please never drop the official course decks as I use these to familiarise myself with the words in each skill before I work through the lessons. What is needed please is updated official courses to match the revised trees such as Spanish and Portuguese. The curriculum on the Schools website matches the new trees but not Tinycards.

August 14, 2018

https://www.duolingo.com/JustinSensei

That's helpful feedback - thank you!

August 14, 2018

https://www.duolingo.com/dogomolo

But how do you add small button to Android screen? Maybe other design should be used on mobile screens

August 13, 2018

https://www.duolingo.com/Jimwatana

Which courses is this available on? Great idea!

August 14, 2018

https://www.duolingo.com/JustinSensei

Initially, this will be available for three UI languages (English, Spanish, and Portuguese) and several learning languages (Catalan, Welsh, Danish, German, English, Spanish, French, Italian, Dutch, Norwegian, Polish, Portuguese, Swedish, Turkish). If the experiment is successful, we will look at adding more courses in the future.

August 14, 2018

https://www.duolingo.com/Yoreni4

iam so unlucky on these a/b tests

August 15, 2018

https://www.duolingo.com/Epicity

Just wanna say, that this was an awesome feature. And now my testing is over, i'm at a loss.

August 20, 2018

https://www.duolingo.com/JustinSensei

Hello! You should still be able to see this feature on Web if you're in the experiment group - and if I'm looking up the right username, I think you are. The experiment isn't running on Android or iOS yet.

August 27, 2018

https://www.duolingo.com/Epicity

Yeah, it keeps appearing and disappearing. What's with that?

September 15, 2018

https://www.duolingo.com/F_582087

I've been studying Russian on Duolingo for the past 6 months, but I just started using Tinycards, because I realized I was not storing enough words in my long-term memory.

Unfortunately, I see that this new feature is/will not be included in the Russian course for now, but I would like to add my little 2cent from a user that just started using Tinycard after being halfway through the Russian course.

Like Duolingo Users who view a hint can have an option to click "Add to Tinycards", I would glad to see in Tinycards an option like that, to include a specific card in your private review deck.

You know, when you review all the card of the lesson, close to the little (o) and [!] you could include an extra option,

"Add to my Private Review Deck"

So if I want to review all the course I'll just to the whole unit, skill; but I want to focus only on my weak point, I'll do my Private Review Deck.

And please, do NOT drop the official course deck, as those are required to study/refresh/focus the vocabulary.

August 25, 2018

https://www.duolingo.com/KarinSumbl

this sounds great! it makes it specific to th ewords each user needs to know. Actually, i thought it was doing this already, because some of the words i clicked did end up in the tinycards.

August 30, 2018

https://www.duolingo.com/KarinSumbl

when i'm typing out the words, i can't put an accent on them b/c there isn't a convenient accent box, like in duolingo. Will this feature be added in?

August 31, 2018

https://www.duolingo.com/JustinSensei

If you practice on iOS or Android, you can long-press a key on your keyboard to see related special symbols, like â, ê, î, ô, û.

For now, on web, you can use special keystrokes to enter these characters with your regular keyboard. For example, here's an article that explains how to type French accents on a Mac: https://macinfo.us/how-to-type-french-accents-on-a-macbook-pro

If you'd like, you can also just ignore the characters for decks that allow typos - those will treat "a" just like "â" when scoring an answer.

We've been looking into ways to make it easier to type special characters on all devices (we recently ran an experiment on one possible approach, and we're looking into something like what Duolingo does) but we haven't found quite the right solution for Tinycards yet.

August 31, 2018

https://www.duolingo.com/KarinSumbl

Thanks Justin. Unfortunately i'm on a microsoft laptop.

August 31, 2018

https://www.duolingo.com/JustinSensei

For a Microsoft device, these tips might help: https://frenchtogether.com/french-accents/

I haven't tried in French, but for German it's pretty easy to use the US international keyboard for things like ä and ß. At least once you get used to it!

August 31, 2018

https://www.duolingo.com/piguy3

Adding an international keyboard is generally the way to go for things like this. Works perfectly well for French, and gives you the accents without the annoying changes in key position for e.g. "a" and "z."

It shouldn't take you more than a handful of mouse clicks to activate.

August 31, 2018

https://www.duolingo.com/Augustine2017

I use Microsoft Windows and use the US International keyboard to type in French, German, Spanish and Italian. For polish I had to download an additional keyboard, but that is also very easy to use.

October 6, 2018

https://www.duolingo.com/The_Mama

I use AllChars. It's a tiny free program that sits in your Windows tray, just waiting for you to hit the control key (all by itself) to activate it, followed by a 2-key sequence to create the character. So I can get almost all the special characters I need, like é è ê ç á í ó ú ß ä and more, but not a couple of Polish characters.

September 1, 2018

https://www.duolingo.com/EricHillia

I just noticed this the other day. I love it!

September 7, 2018

https://www.duolingo.com/JustinSensei

UPDATE: The Tinycards team has decided to shut down this experiment. While it has been popular among a subset of users, on the whole it has had a neutral to slightly negative impact on the learning experience in Duolingo. Going forward, we'll be looking into some other ways we can make it easier to practice specific words from Duolingo, ideally without requiring the use of a separate app.

Note that decks created via Duolingo hints will be available indefinitely, but as we shut down the experiment, you will no longer be able to add new hints to any existing hints decks.

September 10, 2018

https://www.duolingo.com/JenGresham

I'm gutted. The easy ability to add new words to Tinycards and then practice them everyday was helping me so much. I literally didn't want to do a new lesson without the feature. I understand the benefit of creating an option that doesn't require a separate app, and Tinycards was not without issues. But what about continuing the experiment or access until the other option is available? Or what about making this a paid addition for those who want it? Since the tech exists, seems a shame to remove the access for those who loved it.

September 11, 2018

https://www.duolingo.com/Thomas.Heiss

@all

Lekz had made an Android app to review up to 99 flashcards based on the old Duolingo API which was available as the "Review flashcards" button on the "words tab": https://forum.duolingo.com/comment/19154823/I-ve-made-an-Android-app-to-review-learned-words-same-as-in-Review-Flashcards

As it is based on the old Duolingo API, it obviously has the same problems as the Python 2016 "Review flashcards" button that it uses RANDOM words and does not heavily focus on the very weak strength bar 1-3 words in our words list.

Generated German noun words flashcards also seem to miss der/die/das prepositions.

Well, better than nothing the one or another user might want to say...

September 11, 2018

https://www.duolingo.com/JustinSensei

I count myself as one of the users who really liked being able to add hints to Tinycards, so I understand that it's tough to see the feature go away. But we haven't lost sight of the need for something like this - we just need to find a way to do it so that it benefits more users more consistently, and does so with fewer downsides.

September 11, 2018

https://www.duolingo.com/JoesCathbert

How can it be a downside to others? Because only those who want Add to Tinycards will use the feature and for the rest it is not compulsory. Do you think this affects them? I look forward for your answer

September 13, 2018

https://www.duolingo.com/EricHillia

I think what Jason is saying is that the data suggests retention for the majority of users is not improved, and possibly lessened in some cases. I personally noticed the opposite, but I'm an anecdotal dataset of 1. :)

September 13, 2018

https://www.duolingo.com/JustinSensei

Eric has it right - when we looked at the data, we saw a net negative impact on Duolingo users, even though the qualitative feedback we got was almost entirely positive. This happens sometimes, and it's one of the main reasons we run these experiments. When it does happen, it usually means we need to do some more research and some more testing to figure out why an experiment had a negative impact that we didn't expect.

September 13, 2018

https://www.duolingo.com/MetalFatigue82

Bummer...Actually this was the only reason I installed tiny cards. I felt like a good complement to duolingo. Uninstalling tiny cards

September 13, 2018

https://www.duolingo.com/AmyBygler

How does having a way to create appropriate and personally customized flash cards have a negative impact on learning? It has helped tremendously already. Being able to pick out particular words that were problematic to practice had an immediate positive impact on my retention, allowing me to recall much more automatically and therefore focus on the grammatical aspects of my learning. I personally feel removing this feature is a BIG mistake on the part of Duolingo. So, so disappointed.

September 14, 2018

https://www.duolingo.com/JustinSensei

I share your frustration! I also really liked being able to pick individual words to practice more, and I think that's a feature most language learners would benefit from. We're still looking into ways to put a feature like that in Duolingo.

The problem was that, the way we implemented the feature in this experiment (ie, through an integration with Tinycards that pushed vocabulary practice into a separate app) didn't work particularly well. The experiment suggested pretty clearly that a lot of people want this kind of feature, but when they tried this version of it, many people became frustrated or confused or distracted enough that they engaged less (and learned less) overall. It's something that definitely worked well for some users, but on the aggregate it wasn't an improvement on the status quo.

September 14, 2018

https://www.duolingo.com/qomanaut

I think you are making a mistake most companies are doing these days. Although you figured out how to measure the outcomes of an innovation, you only look at the mean/average. Your customers are probably a very heterogeneous group of individuals (students, adults, people who seriously want to learn a language, and people who just do it for fun). Just looking at the average does most likely not provide you with the correct information about your "experiment". It is important to evaluate subgroups. In the end, it might be wrong searching for the tool that fits everybody (on average), rather than a combination of tools that fit different types of individuals. As far as I can see, this tool helped a particular group of your customers. It might be more rewarding to look into an additional tool that supports those learners who performed worse with this new feature than to abolish the feature and look for a completely new one.

September 19, 2018

https://www.duolingo.com/EricHillia

Justin, you are truly breaking my heart. As someone who uses both apps every day, I thought this was one of the greatest updates that had been made recently. I actually deleted my personal deck that I was manually creating because this quicker, easier process was offered. Not to mention I was really seeing much better retention with new conjugations. I understand why, but personally, I'm very sad to see this go. :(

September 10, 2018

https://www.duolingo.com/anka_sirota

Would Tinycards team consider enabling this feature for Duolingo Plus users? While we can only guess what their metrics of "learning experience" are, surely paid accounts can be reasonably exempt from at least some a/b testing.

September 15, 2018

https://www.duolingo.com/Kanapepa

Very sorry the hear it, really liked this service.

September 20, 2018

https://www.duolingo.com/intricatecakes

Very sad to see the integrated experiment go! It made creating Tinycards decks much easier. Making them manually takes too much time, so now I just don't bother. Hope to see something similar sometime!

September 14, 2018

https://www.duolingo.com/SinbadJCol

Damn! I agree this was one of the best features of Duolingo. It really helped some words just dont seem to sink in and this really helped to make it work.

September 14, 2018

https://www.duolingo.com/dogomolo

Most importantly how this feature could be implemented on Android/iOS. As rev._mother commented users can find it annoying when too many cards are added to the special deck. Users should have enough control to build their decks properly.

I have one idea. Include a slider in every lesson that decide if hover saves words or not.

And then what? I wonder how complicate you make practicing words I have saved to my deck. Where is my deck located? On opening screen or what? How I decide if I go to practice my saved words or other words in a course deck? Do I have to always make that decision when I open Tinycards?

August 11, 2018

https://www.duolingo.com/JustinSensei

Users who view a hint will need to click "Add to Tinycards" if they want to create a flashcard from the hint, and all the hints added for a language will be put in a special private deck for you. That deck will automatically appear on the Tinycards home page.

We're already working on plans to bring this feature to Android and iOS if our initial experiment on web is successful. So stay tuned!

August 13, 2018

https://www.duolingo.com/EricHillia

Hi Justin, off topic, but... It appears that my bar is going down whenever I try to strengthen a deck. It will go from almost full to empty after completion.

September 21, 2018

https://www.duolingo.com/JustinSensei

Hi, Eric! Let me ask the team and see if we've heard any other reports of this problem. Thanks so much for your patience! In the meantime, are you using Tinycards on the web, on iOS, or on Android?

September 25, 2018

https://www.duolingo.com/EricHillia

Hi Justin,

It appears to be an issue on the Android app. I haven't tried my iPhone or on the web. It happens when using the strengthen option from the main menu, but not if you go into a deck first. At least, I haven't had it happen from the individual decks.

September 25, 2018

https://www.duolingo.com/JustinSensei

I have a guess as to what might be happening. Do the bars animate when they decrease? Or do you just start studying then notice that the bars are lower when you're done?

September 28, 2018

https://www.duolingo.com/EricHillia

The bar is almost full, and then it animates down to almost nothing.

September 28, 2018
Learn a language in just 5 minutes a day. For free.