Is the literal translation, "Why me not you visit?" And is this grammatical structur similar for these types of questions? I have a rudimentary understanding of cases.
Well, given the case system, the position of "mnie" won't make the sentence ambiguous. That's one thing.
Anyway, the reason that "mnie" is where it is here, is because the second possible option is "Dlaczego nie odwiedziliście mnie?", which isn't the greatest, because if only you have the possibility to avoid putting a pronoun at the end of the sentence, you should indeed avoid it.
If you shouted "You did not visit me!", that's "Nie odwiedziliście mnie!" and there's no other place to put "mnie". You don't really start a sentence with a personal pronoun in a different function than being a subject of the sentence.