Yes, personally, I prefer that: "la couleur n'est pas importantE" or "la couleur n'a pas d'importance".

It would need some complement to stand straight: "la couleur ne fait rien à l'affaire"

"ça ne fait rien" is used as an impersonal phrase like "it doesn't matter", but with a real subject, the phrase loses its meaning.