Then I guess 'noc' is feminine, right? So why is 'good night' translated as 'dobrou noc' instead of 'dobrá noc'?
Yes, noc is feminine.
I usually explain it like this: dobrou noc uses accusative dobrou because it's a shortening of Přeji ti/vám dobrou noc. 'I wish you (sg/sg formal+plural) a good night.' You use accusative for the object of a verb.