Can someone explain to me why we use उसे here and not वह?
चाहिए is always used with the dative form of a pronoun (उसे, मुझे, हमें etc) or a noun+को.
Think of it as being literally translated as 'To her, fever medicine is needed'.
Have we discussed nouns modifying other nouns yet? Is the rule here just to use the appropriate form of ka? E.g.:
work clothes = kam ke vastr
butter knife = makkhan ka chaakoo
I wrote उसको बुखार की दवाई चाहिए and it wasn't accepted. Isn't it the same?
It should have been accepted. You can report if you see the sentence again.
why is it bukhar ki davayi and not davayi ki bukhar? Doesn't "fever" describe the kind of medicine she needs?
Think of it as 'Medicine for fever' or more literally, 'Medicine of fever'. But since Hindi uses postpositions instead of prepositions, the order is reversed.