Since 'babci' is both singluar genitive and locative, could this mean both "we are walking to our grandmother" and "we are walking to our grandmother's [e.g. house]"?
It can be "going", true. And obviously it works. But if you used "driving", then you sure need to use "jedziemy" in Polish.
dam I wrote "grandma's place" and it corrected me to "grandma's flat"... then I tried "apartment" but that didn't work either lol