This is a perfectly normal statement to hear from a Chareidi (Hasidic or Yeshiva) Jewish man
Is this a setup for a joke, or are you being serious?
I would assume it is because touching a woman during a certain time would render them unclean, and since people rarely advertise such things... They play it safe.
No, I'm being serious.
The reason they do it is because they believe touch between opposite genders to be a powerful, sacred thing, and to be reserved only for contact between a husband and wife. They claim that refraining from touching women whom they are not married or related to, combined with obeying the Jewish laws of family purity¹, can produce a greater degree of sexual pleasure and intimacy, as well as keeping romantic feelings fresh throughout the marriage. For this reason, even young people who are not married yet refrain from touching the opposite gender, in order to enhance the initial contact with their spouse once they are married.
¹This may be what you were referring to when you mentioned unclean, by the way.
As a "yeshivish Jewish man," I would explain it a little differently. It is not that mere touching is such a powerful, sacred thing, but rather it is a safeguard from engaging in actions that involve more than mere touching. To use a phrase I learned in law school, there is a "slippery slope" that begins with touching and can sometimes lead to much worse. The Torah frowns upon promiscuous behavior, and banning touching between the genders is meant to help nip such behavior in the bud. And to use another phrase from law school, a ban on touching is a "bright-line test" - easy to understand and implement. No need for an "I know pornography when I see it" standard, which sadly doesnt seem to be as clear cut as it should be.