Haber represents 'have' in the sentence. If you weren't talking about what you could have done the haber would be in the él form (ie ha ido, he has gone). Instead you could have done it so take the infinitive Haber.
I would absolutely say something like this. Suppose someone said, "Where could he be? Could he have gone to school?" My reply would be" No, it's Sunday. He couldn't have gone to school. Why would he go to school? The school is closed."
my exercise was "type what you hear"; what I don't understand is why "no pudo 'haber' ido...." ? y non "no pudo ir a ....." ?? ( not native speaker in both languages ) The question is about the Spanish verb, not the English translation.