My translation was literal based on what I knew žena to mean from the lessons. In English, at least in slang/informal speech, endearing phrases like "my man" or "my woman/girl/lady" can mean things like "my friend, my boyfriend, my girlfriend, etc." (but usually not "my husband" or "my wife").
But it sounds like in Czech, or at least in this context, žena, can ONLY mean wife, which is good to know.
I agree in this simple context it should be 'wife'. Moreover, this is a Czech lesson, not English. However, throughout life and literature, man and woman have been used as husband and wife, whether actually married or not. "Don' nobody mess with ma woman!" And that song: "Stand by your man!"