I think when we have no other context before this sentence HIS is going to be understood as what "su" means here.
De acuerdo, tiene razón. ¡Gracias!
This sentence could also mean: "He added that YOUR house is big."
i believe el agregó... sounds better in spanish.