Translation:You are the neighbor who swims in my pool.
Seems like someone who never leaves the house but allows his neighbor to swim around.....then they meet one day.
Or maybe he noticed that someone was swimming in his pool while he was somewhere else and he finds out it's his neighbor. Later on they meet and he says that.
The subordinate clause beginning with che is referring to "the neighbour", not tu, and so it has to be inflected for the third person.
This recalls an intriguing Burt Lancaster film from the late 60's entitled 'The Swimmer'. In the midst of a breakdown, he strolled the neighbourhood doing just that.