Could "be inte mig" be correct here, in the sense of "do not ask me to do that"?
Yes, although I doubt that many native English speakers would phrase it that way.
Okay thanks! :)
Are "Fråga mig inte" and "Fråga inte mig" both correct? If yes, which one sounds more natural to native speakers?
They're both correct, but they're likely to be interpreted slightly differently. While both essentially mean "I have no clue", if you use fråga inte mig it places the emphasis on the me, often (but not always) implying that somebody else might know.
Oh, ok. Tack!