Don't think so. It's probably random. If you want to associate a meaning to it, there is the belief among certain people in Korea that cheating partners choose lonely and private hobbies to have more freedom to act on their urges, e.g. hiking. And fishing maybe?
Why the need for a plural particle with "women" and not "husbands"? Is it to indicate the singular possession of each individual in the group of women (1 husband per woman) or is it simply because Korean is a pro-drop language?
남편 here is general (referring to any/all husbands who fish), so the plural form is implied and 들 is dropped. The plural particle is added to 여자 to clarify that the sentence is not referring to a certain woman in particular, but multiple.