The imperative works in two different ways. "Non andare" is for a single person, your son, your cousin, your friend, but it shows up on public signs as well. "Non andate" is for addressing a group, just like you said.
But I was wrong about No, non vai which should have been No, non andare (infinitive for negative imperative as you say) but Si, vai is correct. But now what about Vada a bordo which, famously, means "Get back on board". But how?