麻烦 means troublesome, not annoying. I've never seen 麻烦 being used in the context of annoying.
I do not think annoying is the right wording
Bothersome, or maybe annoying yes, when it comes to describing people.
Isnt "wo de laoban hen fannao " a better translation?
That would be "our boss is annoyed / troubled."
Duolingo marking correct answers wrong is as well.