Duo should accept "My husband's family." No native speaker says "the family of my husband" does not sound natural, not even a little bit.
Why can't you also say, "The family from my husband?" Meaning from my husband's side?
Because the preposition "from" indicates origin (in the sense of the source -be it a place or a person- where something/someone comes from), not 'ownership'.
Apart from that, we don't say "from" my husband's side, but "/on/" my husband's side.
Why isn't the family of my wife allowed. How do you know that it's the husbands family as echtgenoot simply means spouse.