You are right that literally is means good luck, but if you want to wish someone good luck (for example before a test or a performance) you would use בהצלחה.
I have sometimes heard "mazel tov" instead of "mazal tov". What's the difference? Could it have something to do with the male/female form or is it just pronunciation?
Mazel tov is the pronunciation in Yiddish which is related to Hebrew but actually a distinct language
תודה רבא. I'M lear Hebrew but grew up with some Yiddish. Thank you for clarifying that finally.