Translation:It is not clear whether the shirt is too big or too small.
Why do you have to add "too", while the previous identique sentences where rejected if translated with "too"? It seems to me that just because the word "too" is in the list you are expected to use it; anyway this is incoherent and generates confusion and loss of time and confidence in the reliability application.
I believe inversion in this case is added for emphasis (Непонятно что? Велика или мала рубашка.) It won't be a mistake to say, 'Непонятно, рубашка велика или мала' but will sound slightly less natural. It is important to bear in mind that the Russian или is more versatile compared to 'whether' in English. There are many examples where you don't inverse the word order and use или as or.
It's common to say it the way you suggested (Рубашка мне большая) though grammarians would probably cringe. Большой denotes that something is big compared to a similar object of the same kind (this is a big shirt and that is a small shirt). Велика/мала are used to indicate the shirt's size in relation to its wearer. Мне эта рубашка мала. А тебе как раз! (This shirt is too small for me but just the right size for you!)
I don't get how мала translates to "too small". мала means "a little". To describe a shirt that is too small for you, wouldn't you say рубашка маленькая or рубашка мне не подходит? And what's with велика? Never heard that word to describe a shirt lol. How about instead рубашка на мне мешковатая or рубашка на мне маленькая/короткая?
In all the sound exercises in this part https://www.duolingo.com/skill/ru/Short-and-Long/ my recordings are supposed to be incorrect. I did very well in the 36 lessons before this one. I am afraid Duolingo has not enough capacity to help me out. I live in the Netherlands and I hope it is not true.