Translation:We go to the house of my brother.
Why is casa only used for house and not home?
Well it's used as "Estoy en casa", the same way as we say "I'm at home". Home is Hogar
He makes an (el) sound before hermano. Its incorrect but also misleading.
Let's go to my brother's house is also correct