"She does not wear a red hat, but a green one."
Translation:Sie trägt keinen roten Hut, sondern einen grünen.
Is "einen grünen" male accusative
Masculine accusative, yes.
Is "grün" when used as a noun not neuter?
it’s not a noun; it’s still an adjective.
In English, you need a dummy noun “one” in this situation but in German you can just use the adjective in the form it would have if the noun were there — thus einen grünen Hut “a green hat” becomes einen grünen “a green one”.
She doesn’t have “green” (the colour).