Is me or in zimna the i modifies the sound of z to ż, like it does in się from s to ś?
I'm not good with explaining phonetics, but you are generally right (but it's Ź - with an accent, not Ż with a dot).
I would simplify it like that: zi is pronounced like źi, and si is pronounced like śi.
si, zi - they are always or at least almost always palatalized. Compare between Polish and Russian:
The word looks the same (apart from different alphabet), but the pronunciation is quite different. For a foreigner, I guess the Russian pronunciation may be more logical.