-eur -> -euse/-rice?
Le chanteur -> La chanteuse
L'acteur -> L'actrice
Is this just something to memorize, or is there a rule/reason behind whether a masculine word ending with -eur becomes -euse over -rice?
unfortunately, it is just to memorise. I don't think there is any rule...
There is indeed a rules, but most French speakers are not aware of it!
-euse is for adjectives and for nouns directly derivated from an action verb with a t in it: chanter -> chanteur / chanteuse mentir -> menteur / menteuse manger -> mangeur / mangeuse
and in general for nouns of popular origin that evolved "naturally" in everyday language
-trice is for when the verb whose radical form does not include a t and the derivation occured in the Latin form before it entered into French:
agir -> acteur / actrice faire-> facteur / factrice diriger-> directeur / directrice éduquer -> éducateur /éducatrice
Finally there are a few rare forms in -eure : auteur / auteure (autrice was a short-lived attempt in early 20th century) successeur / sucesseure (but many French people would be unsure and rather try a periphrase)