You are right, there is no way to know in this sentence. In fact, I would understand this sentence as "nobody talks about their clothes". In Hebrew we have a type of sentence called משפט סתמי, which is a sentence with no particular subject. It can be translated using the English "one" (one does not talk about their clothes), the German "man" or Spanish "se". It's a bit similar to passive sentences in English, for example:
לא רואים מכאן את המגדל -The tower cannot be seen from here
or - one cannot see the tower from here.
(The Hebrew passive, less frequently used sentence would be המגדל אינו נראה מכאן)