Yes, but not when it is before a noun.
Before a noun, an adjective has to take an ending to show gender/number/case -- so "my small brother" is mein klein·er Bruder while "my small·er brother" would be mein klein·er·er Bruder with one er for the comparative and another er for the masculine nominative singular ending.
However, mein Bruder ist kleiner for "my brother is smaller" would just have the comparative -er ending, since in that case it is not before a noun.
Further to the point made by lesgle, something has to show the gender of Bruder if adjectives are present.
Had there been an article such as der to match Bruder, the adjective could have followed its own rules. But because the article replacement is an ein word (mein) in this example, it doesn't show the masculine gender.
nominative indefinite article
ein, eine, ein, = m.f.n.
The duty of showing the masculine gender then falls to the adjective. Instead of its normal kleine spelling that would apply, it must now show the masculine nominative er ending that would have been displayed by the definite article which does show the gender.
der, die, das = m.f.n.
Thus .....kleiner. Since kleiner already has the er ending, if you decide want to make it plural you have to add another er ending
It is a dirty job but someone has to do it if adjectives are present. Because ein words don't display the masculine er ending the adjective has to.
I think it is better:
Ich bin zwanzig Jahre alt - I'm twenty years old
"Mein kleiner Bruder ist zwölf" also works. It's what I entered and was correct.
Why do you think it could be kleinen?
It's the subject of the sentence, so it's in the nominative case.
It's after mein, a possessive adjective, so the adjective takes mixed inflection.
Mixed inflection for masculine nominative is the same as strong inflection, i.e. -er -- the word before it has no ending (mein rather than meiner, meinem etc.) and so the adjective has to show the gender, number, and case.