I will look it up to see what Allen is precisely saying, but I did read him a long time ago, and the way I remember it is that the shift from ph (an aspirate) to f (a fricative) in the common or colloquial language happened no sooner than the 1st c. AD. If that is the case to say that it was pronounced one way or the other in Classical Latin (or by the Romans, as someone else argued here) is a bit ... misleading. The vacillation, which preceded the actual shift, starts around before AD 100. Cicero, for illustration, was born 106 BC, and Caesar was killed 44 BC. So to make claims about the "classical pronunciation" based on some Pompeian street-talk graffiti from AD 79 is not quite fair. Even as far as the Greek pronunciation itself is concerned, the educated people as far east as Egypt or Armenia, pronounced their phi's aspirated until as late as the 6th. c. CE. So again, the most educated option and indeed the least common denominator here would be to render these aspirated.