No harm at all. You can do it as troll said. It is good Greek both. And more: Ποτέ δεν τον έχεις ακολουθήσει; in a negative interrogative form, that means in free translation "are you sure you have never followed him?". But it is wrong Τον ποτέ έχεις ακολουθήσει;
The word order of Greek is flexible in general, but this one doesn't work. If one changes the accent mark on "ποτε", it would make sense: "Πότε τον έχεις ακολουθήσει;" and would mean "When have you followed him?"