Translation:He had already left the party at that time.
Well, because that really doesn't make sense, does it? 'had already' is talking about something that happened at some indistinct time in the past. 'at that moment' rather emphasizes the exact point in time, which seems irrelevant to the sentence.
My question is why it should not accept 'by that time' instead of 'at that time'. I think that is the sense of the French sentence, but I could be wrong.