how am I suppose to make it out that that's the plural "you" and not the singular "you?
Doesn't matter - it accepts both.
No it doesn't. Just plural
It accepts both. Singular you: Nem hallod?
Plural you: Nem halljátok?
(note that this is the definite conjugation. )
Singular and plural
Te nem hallod is also an accepted answer. What's the difference?