«Чей этот дом?» would be a different sentence grammatically. «Чей э́тот дом?» is a question with an answer «Э́тот дом мой», «Чей э́то дом?» is a question with an answer «Это мой дом».
It's not a possessive adjective, it's the same «это» that is used in «Это мой дом». When asking a question, we move the question words to the beginning of the sentence, but we keep the grammatic structure.
Чей never means 'what?', it always means 'whose?'. Not just in this sentence, but anywhere.