"the letter gh"
You are correct. It would be ideal if Duolingo never changed the capitalization, but it was not designed with Klingon in mind and sometimes recapitalizes our carefully crafted Klingon entries. The programmers are supposed to be fixing it, but Klingon may not be a high priority language for them and we don't know when they'll be getting to it. For a summary of the known problems, check out the thread at https://forum.duolingo.com/comment/26778885
At least the names of the consonants do.
The names of the vowels start with a glottal stop and end in "t", e.g. 'et for the name of the vowel sound e.
For the consonants, it's a bit like how most of the English consonant names end in "ee", e.g. "bee cee dee gee tee vee" (but not all of them, e.g. kay, em).
I have one user report that "if i type say or ray it told me that i speak in English." I've seen such a message before, if I accidentally translate a phrase on a listening exercise rather than just transcribe it, but I haven't been able to reproduce it. Has anyone misheard ghay as ray or Say (fully possible) on a listening exercise and been told they typed in English when they answered?