How can someone know that "Look at that house" is said to more than one person?
just with a context, otherwise there is no clue.
couldn´t it be "olhe aquela casa" ?
yes, for sure!
So in Brazil, olhem is the imperative for olhar? In my grammar book it says that the imperative for olhar is olha-olhai. Is that just for European then?
olha, olhe, olhemos, olhai, olhem... they are all imperative! It will depend on the pronoun it is being referred to! olha, olhe (you-singular), olhai, olhem (you-plural).
Ok, so olha/olhai is for tu, and olhe/olhem is for voce? My damn verb book only lists olha and olhai, it must be written for European where they don't really use voce?
if you need any help with verb conjugations, you can check this website out! It's very helpful. Try to solve your doubt by looking at "Imperativo" http://www.conjuga-me.net/verbo-olhar
That is a great website! Thank you for sharing that link.