"Are we wearing caps?"
That may be. Yet, looking at you, and what you are wearing at the moment, nobody would ask "Mit hordasz?". They would ask "Miben vagy?", "Mi van rajtad?".
The "hord" version would usually be used for "We wear hats in the winter" - "Télen sapkát hordunk", those kind of sentences.
Or they could look at the beautiful hat that you are wearing at the moment and ask "Do you always wear this hat?" - "Mindig ezt a sapkát hordod?"