I have the same question. Asking questions with 那儿 seems to be the same as making statements with 那儿? I.e. in these examples, "Is the bathroom there" = "Is there a bathroom there?" I'm used to looking out for the "ma" ending for questions now, so this is confusing.
You have your here/there mixed up 那=nà=there, 这=zhè=here. Im not sure on the direct translation of this sentence but im imagining this is in response to a question like 医院在这儿吗 (Is the hospital here?) And a response could be 不，在那儿 (No, it is there)