I am interested in this as well. It seems "off of" is not accepted; I did the same thing in another exercise and it was also marked wrong. Maybe it takes adding another word to explicitly say "off of?"
Maybe they are not “our” hats. Maybe they are stolen hats. Maybe they are part of the teacher’s collection of hats that he brought in to show off to the class and now wants them back. We may never know.