It's the same question as with "already". From what I've heard, slang and spoken language aside, "already" and "ever" have to be used with perfect tense... "Have you ever been" and not "were you ever".
Has it changed now, and simple past is allowed as well? How about "Were you ever to Denver?", is that OK too? Thanks!