that is not correct. some colors have gender. "A camisa vermelha" "o saco vermelho". Other colors that do have gender: amarelo(a), roxo(a), branco(a), preto(a)
the colors that do not have gender are those colors that derive their name from other sources, like "laranja" comes from the fruit (orange), cinza (gray) comes from cinzas (ashes), rosa (pink) comes from the flower, and so on. Some others who do not (as far as i know) come from other words but also don't have gender: azul, verde, marrom.
It's tough i know
In this case wouldn't you actually use "laranjado" instead of "laranja" so this should read: Ela veste um vestido laranjado. No? For instance I could ask you what the color of your dress is, and that color is "orange" ("laranja"). But if I were to point out the color of someone's dress, I would say that her "dress is the color orange" ("laranjado"). The subtle difference between the two being that the former refers to the pure color orange, while the latter characterizes the dress to have the color orange, hence "laranjado."