Why not az anyátok?
It works both ways, with or without "az". I don't know, it may be a special case for relatives. Or "anyátok" is treated like a proper noun. Something like that.
"Anyátok" here could be rude, why not have "édesanyátok" or "anyukátok" instead?
"anyátok" isn't rude, the "az anyátok" expresion it is.
First we learn those forms are possible:
a ti autótok
a nektek az autótok
And now, with family, a proper form is even rude???
But "a barátnőd" is not rude?
a barátnőd is not rude. Az anyátok / Az anyád can be consudered rude, because some people use it as a short version of "a k..va anyád" (and that middle word is rude)
"anyád" can sound rude regardless of the article, probably "anyátok" is used much less commonly as swearing ("anyátokat", though...)