I like your reasoning, it makes intuitive sense. However, we use ett as in ett, två, tre ("one, two, three") only if there is no noun specified. If there is a noun, the "one" is treated as the indefinite article en/ett instead.
Why isn't "Det är en bara sak" correct? To me it makes sense. For example, in Spanish it is the same to say "Hay una sola cosa" and "Hay solo una cosa". Doesn't Swedish have this flexibility at the moment of making sentences?