Using fresh to describe an inanimate object in English requires a very narrow context to avoid looking strange.
Fresh is sometimes used figuratively to describe abstract things that have lifelike qualities in the sense they can grow, deteriorate or even die out. Evidence, ideas, approaches, even a look can all be fresh or not.
Bringing in a fresh rock for examination actually means the examination is fresh, the rock is unchanged. The shirt is unchanged regardless of whether it is worn or not. It's use is what is fresh or not fresh.
It is possible to torture a context into making fresh acceptable in this example but Duo wants you to know that your use of it this way is wrong for ordinary conversation.