"V Polsku jsme byli pětkrát."

Translation: We have been to Poland five times.

[JohnBraga1]

Would not 'We have been IN Poland five times' be a more accurate translation? Surely been TO Poland would be do Polska?

[endless_sleeper]

Nope, Czech works differently. "Do Polska" implies direction and cannot be used with "byli", as that implies location.
"Do Polska jsme byli pětkrát" is simply nonsense. You can however substitute "byli" (být) with a verb of movement, eg. "jeli" (jet) or "letěli" (letět).