Why is "your" feminine? Is it because "book" is feminine? Would "your" stay in the female form even if I am speaking to a man in this situation?
Yes. When dealing with possessive pronouns, the pronoun has to match the gender (and number) of the object that belongs to him/her/it.
Yes, the book is feminine and hence, "your" is feminine. No, "your" will become masculine if the book is replaced by some masculine noun such as shoe (जूता) and the sentence would be- यह तेरा जूता है|