Yes, I'd say "I don't eat goat" (ditto chicken, snake, kangaroo).
machieng is quite right. It turns out "goat meat", "goat's meat" and "goat" are all correct English to describe the flesh of the animal, so all three should be accepted as translations of "nyama ya mbuzi".
There's a lot of discussion about it in this course - click on almost any question with the word "fruits" or "matunda" in it. But the basic rule is that "fruits" refers to several types of fruit (apples/oranges/bananas/...) in English, but "matunda" can also be used to refer to several pieces of the same fruit (e.g. 4 oranges). When used in that way, the translation "fruits" sounds strange to an English ear.