Yes, you definitely can.
(Disclamer: I'm not a native speaker. But the question is fairly easy so I think I can answer it.)
I am a native Ukrainian speaker and speak English as my second language. The point is, it is ingrammatical to say "I forgot you DON'T eat fish". (However, it sounds perfectly cool in Ukrainian) Sequence of tences presupposes it must be "...you DIDN'T eat fish".
Just grammar folks.