"Chan eil dà bhràthair agam."
Translation:I do not have two brothers.
No. You are missing a couple of things. First, it is 'bHràthair' (lenited), and secondly that's still a singular form. Only 'bràithrean' is plural.
'Aon' and 'dà' both lenite, and even though in English two is plural, in Gàidhlig the noun remains singular when following 'dà'. Three onwards is as you'd expect.
The reason is actually fairly simple, but hard to explain to anyone unfamiliar with the concept of case, number, etc. In short, pairs are treated somewhat specially, and historically even more so.