https://www.duolingo.com/Anneysha7

Information for A/B Tests

I now know what A/B tests are, and know some of them (like, the Immersion A/B test).

I have a question, how many types of A/B tests are there and what are those? Also, what are their functions and how do they impact a user's profile?

May 26, 2016

8 Comments


https://www.duolingo.com/jrikhal

Hi Anneysha7,

Could you relocate this discussion in the “Troubleshooting" forum? Which is the place meant for such discussion. Thx!
To relocate it (\/!\ please, don’t create a new one \/!\), edit your original post then select said forum in the drop-down menu on top-left of the edition area.

Have a look to this short guide to help you knowing where to post your (future) discussions on Duolingo’s forums.


how many types of A/B tests are there

There are several dozens of them: almost anytime Duolingo create a new feature or a ne version of a feature, they first A/B test it.

what are their functions

To test features, that's the purpose of A/B test (note: it's not specific to Duolingo at all).

how do they impact a user's profile

By making that the user will have or not the tested feature.

May 26, 2016

https://www.duolingo.com/Anneysha7

I relocated it! And, thanks for your reply.

A question: What is the Stream A/B test?

May 26, 2016

https://www.duolingo.com/jrikhal

Not sure, probably an test on users' stream (yours is here for example).
Where did hear about "stream A/B test"?

May 26, 2016

https://www.duolingo.com/Anneysha7

Not sure,...I had this question for a very long time after reading lots of posts on A/B tests. So, I just thought of posting it today to know whether I was in any more A/B tests as I have recently lost my Immersion tab.

Edit: I didn't understand what you meant by (yours is here for example).

May 26, 2016

https://www.duolingo.com/jrikhal

"Lost"? You mean that being working on a given course, you had it and now, if you work again on that same course, you don't have the tab anymore?

Edit: I didn't understand what you meant by (yours is here for example).

I mean that the stream of your Duo account is here

May 26, 2016

https://www.duolingo.com/Anneysha7

"Lost"? You mean that being working on a given course, you had it and now, if you work again on that same course, you don't have the tab anymore?

Yes, I had the Immersion tab for French, Portuguese, English (for French Speakers) and Spanish. A few days ago, I noticed that the Immersion tabs were gone from all the languages.

May 26, 2016

https://www.duolingo.com/Portofan

I got this from my users page:

"backcompat_android_29_show_tooltips":"experiment", "new_years_resolution_experiment2":"undefined", "backcompat_android_29_fb_invite_screen":"control", "backcompat_android_27_test":"first", "backcompat_android_28_press_or_hold_mic_speak_challenge":"experiment", "sg_hand_selected_grammar_incumbent_experiment":true, "sg_hand_selected_separated_incumbent_basics_3_experiment":false, "backcompat_android_30_phone_number_registration":"control", "backcompat_backend_notify_integration_v2":"control", "backcompat_test_experiment":"control", "sg_hand_selected_separated_incumbent_basics2_2_experiment":true, "tree_no-BO_en_v2_experiment":2, "sg_hand_selected_separated_incumbent_basics2_4_experiment":false, "backcompat_android_dummy_experiment":"experiment", "backcompat_ios_27_no_op":"experiment", "android_16_fluency_end_screen":"without_fb", "ios_24_name_challenge_copy_test":false, "backcompat_android_passive_aggressive_notification":"take_a_moment", "backcompat_ios_29_streak_repair_test":"control", "backcompat_remove_hardest_daily_goal":"control", "no_op_experiment2":true, "no_op_experiment3":true, "backcompat_compress_user_vocab":"experiment", "no_op_experiment4":false, "sg_use_teaches_lexemes_in_reverse_tap_distractors":false, "ios_25_unified_delayed_signup_test":true, "tree_en_ar_v1_experiment":0, "backcompat_android_30_session_end_runway":"control", "local_grading_with_language_rules_experiment2":true, "backcompat_five_days_of_email_notifications":"experiment", "sg_boost_row_test_challenge_probability_experiment":false, "backcompat_custom_vibrate_notification_experiment":"control", "sg_boost_skill_test_challenge_probability_experiment":true, "android_23_mistake_session_extension":true, "backcompat_comeback_2_thirty_day_reminder":"experiment", "sg_always_measure_learning_experiment":false, "sg_hand_selected_separated_food_3_experiment":true, "backcompat_android_30_self_placement_test":"control", "sg_is_testing_user":false, "backcompat_android_32_save_session_progress_test":"experiment", "android_24_tap_layout":true, "backcompat_android_27_no_op":"experiment", "ios_24_no_discard_progress":true, "backcompat_web_local_grading_regression":"off", "backcompat_ios_test_experiment":"control", "sg_hand_selected_separated_incumbent_basics2_1_experiment":true, "sqrt_skill_strength_experiment":false, "ios_30_gated_skills_test":false, "ios_19_allow_sharing_test":true, "ios_21_history_test":true, "backcompat_web_test_experiment":"control", "web_remove_quiz_experiment":false, "sg_hand_selected_separated_incumbent_basics2_3_experiment":false, "ios_24_register_birth_year_test":true, "backcompat_comeback_3_next_skill_notification":"hi_it_s_duo", "backcompat_notify_return_path":"control", "backcompat_android_30_local_tts_on_slow_network":"control", "android_26_session_end_spinner":true, "sg_hand_selected_separated_animals_2_experiment":false, "sg_test_challenge_probability_experiment2":"0.2", "backcompat_schools_web_assignments":"control", "backcompat_temp_proficiency_monitoring_en":"experiment", "sg_viability_v2_experiment_5":false, "sg_hand_selected_always_counterfactual_experiment":false, "backcompat_backend_new_invite_email_2016":"control", "android_23_async_session_end":true, "backcompat_android_30_show_volume_prompt":"dialog", "backcompat_sg_course_unit_no_experiment_test":"experiment", "android_17_ic_style_lesson_bar_test":true, "sg_always_add_learning_unit_test_challenge":false, "backcompat_web_tc_big_boss_experiment":"control", "backcompat_android_31_congrats_on_registering":"experiment", "backcompat_android_29_post_placement_test_animation":"experiment", "sg_hand_selected_incumbent_grammar_combined_experiment":true, "android_17_use_vector_images_test":true, "schools_classroom_from_language_selection_test":false, "ios_18_use_characters_test":true, "backcompat_android_27_remove_soft_wall_test":"control", "backcompat_five_days_of_email_notifications_fixed":"control", "backcompat_android_30_fb_invite_screen":"experiment", "ios_25_ring_of_fire_test":true, "ios_china_cdn_test":false, "ios_19_fluency_end_screen":true, "android_reverse_speak_challenge":false, "sg_sf_fluency_score_experiment":"control", "ios_disable_china_dns_test":false, "web_no_tips_and_notes_experiment":false, "backcompat_new_chrome_dialog":"control", "ios_24_autoset_timezone_test":true, "sg_hand_selected_grammar_incumbent_experiment2":false, "sg_hand_selected_separated_common_phrases_2_experiment":true, "backcompat_ios_29_postpone_shop_trial_wall_test":"control", "backcompat_android_32_session_end_runway":"large_capped", "sg_always_use_utf8_word_values_experiment":true, "sg_hand_selected_separated_incumbent_basics_2_experiment":true, "backcompat_web_user_search_fade_experiment":"instant", "backcompat_android_29_back_button_ui":"experiment", "web_leave_session_warning_experiment":"control", "sg_hand_selected_separated_incumbent_basics_1_experiment":true, "android_22_local_grading_test":"control", "backcompat_schools_new_topbar":"redesign", "backcompat_legacy_reminders2":"split", "android_speak_challenge_new_ui":false, "ios_no_social_signup_experiment":true, "promoted_retention_test":false, "android_28_local_grading_v2_regression_india":"local_only", "sg_capitalize_judge_options_experiment":true, "android_26_hint_highlighting":true, "backcompat_schools_campaign":"control", "web_noimmersion_experiment":false, "backcompat_android_31_session_end_runway":"control", "backcompat_schools_web_assignments_v2":"show_assignments", "ios_23_local_grading_test":"control", "no_op_experiment":false, "backcompat_schools_practice_skill_picker":"skill_picker", "sg_hand_selected_separated_common_phrases_1_experiment":false, "tree_en_hi_v3_experiment":3, "backcompat_sg_remove_blame_experiment":"experiment", "tree_de_en_v3_experiment":2, "android_26_hard_register_combine":true, "android_18_google_speak_challenge":true, "backcompat_backend_notify_integration":"control", "backcompat_legacy_reminders":"legacy", "sg_hand_selected_incumbent_experiment2":true, "backcompat_android_30_gendered_tts_experiment":"experiment", "tree_es_en_v2_experiment":0, "kc_based_dynamic_tree_progress_experiment":true, "sg_hand_selected_separated_food_4_experiment":false, "backcompat_schools_settings_classroom_code":"experiment", "android_25_no_speak_challenge_on_bad_network":true, "backcompat_backend_bring_back_pass_email_notifications":"control", "backcompat_sg_remove_explanations_experiment":"experiment", "backcompat_new_friend_notify_experiment":"control", "backcompat_android_27_notification_pop":"control", "backcompat_sg_course_unit_no_experiment_test2":"experiment"

So the answer to the question "how many tests" is "a hell of a lot" ;)

May 26, 2016

https://www.duolingo.com/68dustin

Very true.

May 27, 2016
Learn a language in just 5 minutes a day. For free.