There's a fascinating story behind this word! I looked at the Wiktionary page for it and thought the etymology was interesting, so I googled and found this: http://www.haaretz.com/jewish/features/.premium-1.558912
I just clicked on the discussion to talk about the etymology as well :) it is indeed interesting. Since the article is paywalled, I'll basically explain in a few words: תפוז is short for תפוח זהב, i.e. "golden apple". This is taken from the Bible, where it referred to actual golden apples, and not oranges. The two words were later spliced producing תפו"ז or תפוז.
Personal note: When I got my first Hebrew learning book, edited by an Israeli government agency (מט"ח), at the age of 4 or 5 (can't remember), it said that orange was תפוח־זהב. I think the book is from the 1980s, so this form was still considered 'official' not so long ago.