"남자는 자기의 모자를 주웠어요" This sentence can only be translated to "the man picks up his hat" but the one in the question can be either translated to give the last meaning or to "the man picks up the hat" so why is it wrong!!

How do we know if it's "his" or "the"?

This is incorrect, there is no indication the hat belong to the man, he could have picked up any random hat.

couldn't boy be used interchangeably with man?

모자 can mean hat or cap, but the computer will not accept cap. Also, nothing in this sentence says he picked up HIS hat. If he is in a store to buy a hat, the sentence could mean the man picked up a hat.

A man picked up his hat should be accepted since the Korean does not specify the definite article.