"Excuse me, where's our classroom?"
A little surprise for me too this sounds not very natural. Please stick to 在.
It is not wrong though and I would say it for some place I have never heard before and have no idea where on earth it is, e.g. 雷克雅未克是哪里？Where is Reykjavík?
For using 是, my question would be 我们的教室是哪一间？.
Another option is 我们的教室是在哪里的？.
That depends on whether you consider "classroom" as a place or as an object.
I agree with Keith. I usually treat classroom as an object. But the other way is not too wrong either.