Translation:I saw you when you were at my parents' house.
I believe you must use context clues. Typically it is tricky but still evident. When you see something like 'lui e venuto a casa ieri' it's obvious that it is not quite right to translate it as "he has come home yesterday' because it is just trying to communicate the recent past tense. What it really means is 'he came home yesterday but it's so recent so I'm using this grammatical construction'
Imperfetto is usually used for a recurring event, or an event that was ongoing (and certain exceptions to this rule). If it was 'I saw you when you were at my parents house every Thursday', then I think you would use imperfetto, but since this is just a one time thing we don't use it.