Because זאת is not a general "this", it can only come in the context of "this <noun>". Like "this shoe" - הנעל הזאת. To be perfectly fair, I can imagine a context for a sentence like זאת שלך? in the sense of "is this one yours?" but it takes context. Something like: I'm looking for my plate. - Is this one your? אני מחפש/ת את הצלחת שלי. - זאת שלך?

Because "אותך" refers to "you" as an object. When you need a possesive it's "שלך". See tips and notes for this and "direct object" lessons.