That would be the case if there would be a noun ("buntes Haus" would be a case of strong inflection for nouns https://de.wikipedia.org/wiki/Deutsche_Deklination#Starke_Deklination_der_Adjektive ), but "unser" is a pronoun, hence the declination is different, cf. https://de.wikipedia.org/wiki/Deutsche_Deklination#Pronomen
still confuse why unserES is not used here?!
It is "Das Haus", hence it would be logical to use unserES. Right?
Here, we have nomitive declaration for Neutral noun (Haus). In this case, we use unser. unseres is for genitive that we will see later on. Please check the declaration of article unser: https://www.verbformen.com/declension/articles/?w=unser
If you want to use another phrase with the same meaning, you can say "unser buntes Haus" = "Our colorful house" . In this case, we still have nominative, so we still use unser but bunt changes to buntes. To be more detailed, that happens because the adjective bunt follows the article unser and the gender is not clear by the article. Unser is used for masculine and neutrals. So you change the adjective ending in -es.
The same would happen if there were no article at all: "Buntes Haus".