No, the gender of the possessive pronoun matches the one of the noun that is being possessed, not the one possessing. Femenine words in Norwegian can be declined either as feminine or as masculine words. Using a feminine word, such as avis (newspaper), as an exemple, it would be either "Han har avisen din" or "Han har avisa di", meaning "He has your newspaper". Changing the possessor to the feminine doesn't affect the possessive, it would still be "Hun har avisen din" or "Hun har avisa di", meaning "She has your newspaper".
You need to memorise the gender with the noun. If you get into the habit of learning each noun with its indefinite article, you'll be able to tell the gender from that: "en sko" (m), "ei sol" (f), "et hav" (n).
There are a few rules matching endings to grammatical gender, but they only work for select loanwords, so it's best to get into the habit of committing the articles to memory.