What if it's multiple women that are the subject of "our"? Wouldn't it be nuestras? I.e. if this was two sisters talking.
No. Nuestros has to agree with abuelos, no matter the gender of the speaker.
So, it would be nuestras abuelas, nuestras hermanas, nuestras amigas. And nuestros abuelos, nuestros hermanos, and nuestros amigos, no matter who is speaking.