That's simply not true. Trousers existed in Europe more than 1000 years ago.. wearing trousers is something the Romans 2000 years ago criticised their Northern European neighbours for doing. However, you're probably right that the meaning comes from when Mediterranean Romance speakers were still wearing skirts.
All colors have singular forms when they are used with singular nouns, as well as plural forms when they are used with plural nouns. Some colors have gender, and some do not.
For example, rojo(s)/roja(s), roso(s)/rosa(s), naranja(s)/naranjo(s), amarillo(s)/amarilla(s), blanco(s)/blanca(s), negro(s)/negra(s), cafe(s), verde(s), grís/grises, and azule(s)
So, you write, "Los vestidos son rojos." .