"What do you have in your mouth?"

It would.

Still not accepted as of November 27, 2018.

You always need to use «во» instead of «в» (also «со» instead of «с», «ко» instead of «к») when the next form has a vowel dropped that appears in other forms. Since «рот» has a vowel, and «рту» drops it, we have to use «во рту».

When the word has an alternation in modern Russian (рот ~ во рту, сон ~ во сне, у меня — во мне, весь — во всём), we always need to use «во», «со», «ко» with the forms that have a dropped consonant.

(Feel free to skip the nerd stuff below.)

In the past, Russian had extra-short vowels ъ (ŏ) and ь (ĕ)¹. Those got dropped in most cases (ь also altered the preceding consonant, so that's how it became a soft sign), but they became о and e in an only case: when the following syllable had another vowel dropped too. So, рътъ (rŏtŏ) became «рот» because the first ъ is followed by a dropped ъ, and рътꙋ (rŏtu) became рту because ъ is not followed by a dropped vowel.

However, the prepositions въ, съ, къ, объ also had an extra-short vowel. So, if a word with a dropped vowel followed, they became во, со, ко, обо. In other positions, they became в, с, к, об.

All this happened around 11th century. Since then, the rule became a bit blurred: во, со, ко can be put before consonant clusters even when they never had ъ or ь; and some words which used to have a dropped ъ (like пти́ца) are no longer used with во, со, ко.

¹ The usual transcription is ŭ and ĭ, because they come from Indo-European u and i. However, I find it easier to explain why they became о and е if I write them ŏ and ĕ. They were probably pronounced as something in-between o and u, e and i anyway. But if you’re planning to read more linguistics stuff, prepare to see them written ŭ and ĭ. In fact, in many sources they're just written ъ and ь in Latin transcription: e.g. рътъ is transcribed rъtъ.

There are two several differences between your sentence and the proposed correct variant:

• рту vs. рте,
• the presence of твоём,
• в vs. во.

The form. If рот were a completely regular noun, you’d use «*во рте». But it’s not.

Some nouns have two different prepositional-case forms: when talking about location, we use «рту» (во рту ‘in the mouth’, на рту ‘on the mouth’), and in other cases, we use «рте» (о рту ‘about the mouth’).

Another noun like this is «лес» (в лесу́ ‘in the forest’, but о ле́се ‘about the forest’).

So, since you’re talking about what’s located in the mouth, you have to use «во рту».

The possessive pronoun. It just sounds very unnatural after «у тебя». Because «у тебя» already implies it’s your mouth: you don’t usually have anything in someone else’s mouth — that would be very strange. So, «твоём» is not neccessary to convey the meaning: the context tells us enough.

In English, you must precede most singular noun with something: you can’t just say ‘in mounth’, you need to say either ‘in a mouth’, or ‘in the mouth’, or ‘in your mouth’. And of those, ‘in your mouth’ is the most natural option. Russian doesn’t require to precede «рот» with any modifier, except when it’s really needed.

So, ‘your’ is required here by the English grammar, and it doesn’t really add anything to the meaning. In Russian, the sentence sounds much more natural without it.

В vs. во. Some prepositions have several forms (в—во, с—со, о—об—обо). The form ending in a vowel is required when the the following word has a deleted vowel (рот — рту, о is deleted).