I agree with you. If you want to state that the skirt is lengthy then you say "A saia é comprida" ("comprida" is an adjective derived from "comprimento" - length).
If I hear someone saying "A saia é grande" ('The skirt is big'), I would imagine a skirt that is way too large (like 3 or more sizes up). Then I would ask: "A saia é grande em largura ou em comprimento?" ('Is the skirt big in its width or length?').
Some people just don't use the "larga" and "comprida" because they use "grande" to qualify anything that is over the normal size/length.