One of the translations was "we are able to," so I figured if the "non" was there, it would read "we are not able to," and it was counted wrong. It's not right of Duolingo to give a definition and then not accept it.
"... is/am not allowed to" is quite different from "...is/am unable to". My understanding is that potere means you are not able to (cannot/can't) - that is coming from you, not from an external souce (which would be not allowed to.)
I wrote "we are not able". Why is this wrong? It gave "We are able" as a translation of "possiamo" so why is "We are not able" wrong? You can't say "We are not able to" because that's not grammatical - sentences should not end with a preposition.
"possiamo [andare a]....." means "we can [go to]...., or, more literally and I find a helpful way to think about it, "we are able [to go to]...." The infinitive is potere and it's an irregular that one just has to learn.