Why do we mutate brown in this case?
This construction to describe things 'Mae (something) yn (description)' always mutates the adjective following 'yn'
Mae'r llygoden yn frown - The mouse is brown
Mae'r llygoden yn ddu - The mouse is black
yn causes noun to mutate softly, unless it mean in where it causes a nasal mutation.
If llygoden is feminine why does it not mutate to "mae'r lygoden"?
Because the mutation of feminine nouns after "y" is what's called a weak soft mutation which does not affect "Rh" or "Ll".