"Is the restaurant there?"
Yes, it is needed.
It is a common mistake to mix up 哪儿 and 那儿. 哪儿 means Where while 那儿 means There. As here is a Yes-or-No question, 吗 is needed.
You're thinking of the eternally confusing 哪儿，rather than 那儿。note the extra box. I believe 哪儿 is third tone, while 那儿 is 4th