You could say "pants pocket," "trouser pocket," or "trousers' pocket," but 'pant pocket,' while it sounds good, is unfortunately incorrect English. The error is very small, but pants is not a divisible noun when referring to what many call trousers or slacks. You cannot have one pant, two pants, three pants, ... You can only have one pair of pants, two pairs of pants, etc. Thus, albeit confusingly, it's pants pocket. "Pant" by itself is what a dog does when the weather is very hot. Remember that English is basically three-and-a half languages smashed together, so there are a lot of overlapping words.