"My little brother is bad at swimming."
There's no such rule. Here は and へた acts as "is" and "bad" and およぐのが is just "at swimming". およぐのが, which is the direct object, will come after the subject, which is why the は is in front of the が in this sentence.
I'm definitely not an advanced Japanese speaker but I'm pretty sure what precedes が is the subject here. The の is there to make a noun of the verb "to swim" so that it can be a subject. The は is not a subject marker, it's a topic market and can replace or be combined with many particles. I don't know if there's any rule stating it HAS to be the first particle in the sentence but it sure very often is. So to break down this sentence literally (for the sake of my explanation, not for practical proposes) my coldest guess would be: Regarding my younger brother, his swimming is very bad.
Regarding that use of の: What is the difference between using の to make the verb a 'noun' of some sort and こと? I know こと would (if used instead of の in this sentence) translate literally to 'The thing of swimming is bad'
I also thought が is the subject marker. Isn't it that the little brother is the subject here?