Saying "Wir haben er" is like saying "We have he" rather than "We have him". It's the same with "it" (es/ihn), but we don't have an equivalent for that in English. The proper terminology is that "er" is in the "nominative" case (the one performing an action) whereas "ihn" is in the "accusative" case (the one being acted upon).
To have something/someone (direkt object) = ihn. But ihm is the indirect object, e.g. when you give something to someone. English does not have different conjugations here, it is 'him' in both positions, but the indirect object often need a preposition to clearify ( give 'to him').