她们有三个弟弟 was not accepted. I guess they can only have 3 little brothers when there's at least one guy in the group. reporting it
Probably a mistake, better report it!
This translation is correct, and is probably more common as well, but the word "口" as a counter should also count (as 口 is a counter of family members and relatives, so it should be accepted as well). 他们有三口弟弟 should be accepted as well.
口 ia used for the family in general, but saying for example "I have 3 younger brothers" has to have 个. I did it that way (with 口) in class and my teacher said that it is not right.