"I have a sister, but I do not have a brother."
Translation:У мене є сестра, але немає брата.
No, «я маю» is in fact very common both in colloquial and formal speech. I don’t have statistics, but my hunch is that they are roughly equally common.
Сестри́ is the genitive case of the word сестра́, but after ма́ю, you use the accusative case: сестру́.
Брата is both genitive and accusative, but сестра has two separate forms.
(In some dialects, сестри can be used as accusative plural — the standard Ukrainian form is сесте́р for this. So if I heard your version, I’d think you wanted to say ‘I have sisters’, not ‘I have a sister’.)