No, in this case, it's correct. "To heat up" is a phrasal verb, and you can split them as long as they are transitive and you do so ONLY around the direct object. In this case, that would be "the fish", so it is, in fact, grammatically correct to say "We heat the fish up." as well as "We heat up the fish."
You could also say "We heat the fish up for dinner." but you couldn't say "We heat for dinner up the fish." because "the fish" is the direct object still, and "dinner" is the indirect object.
There can be some contexts where it just doesn't work, despite being a transitive verb and a direct object (such as "I stand for the national anthem" or "you will get over the flu"). Similarly, a long phrase, even if it's still the direct object, can make it unclear and so it's not recommended to split the verb then, and there maybe some subtle enhancements or changes to the meaning, or how much it's effect is emphasised, as per this post (https://magoosh.com/toefl/2016/more-rules-for-splitting-phrasal-verbs/) but that's the general principle.
Tl;dr - yes it's correct for this particular example but not necessarily so for all phrasal verbs.