"Why were you looking at her like that?"
Translation:Proč ses na ni tak díval?
In the past tense, 2nd person singular, when you also have to use the reflexive pronoun, then there is only the contracted form correct. That means "jsi se" (and "jsi si") make "ses" (and "sis" respectively).
If you used the plural "Proč jste se na ni díval takhle?", your sentence would be accepted.
The verb "podívat se" is perfective and therefore used for quick, short, finished, one time actions. When you "are looking", it is most likely longer than that. It is much better to use the imperfective verb (dívat se) since it describes slow, long, unfinished, repeated actions.
Can you explain to me why this is incorrect: Proč se jsi na ni tak díval?
I thought that enclitics like (se) always had to come second.
You're perfectly correct about the second position, but the pronoun "se" is not the only clitic in Czech language. When there are more of them in one sentence (like in the sentence you've written), you have to be careful and order them, since they can't all be in the second position.
Look at the link , if you scroll down a bit, the order of clitics is nicely written there with examples.