Nem szegények vagyunk - Would this word order work?
Yes, it would.
The only difference is that it negates the adjective directly. You could continue it with another clause that states what you are instead: "Nem szegények vagyunk, hanem gazdagok"
So in this case dropping the verb to be is not acceptable?
Yes, because it's not van or vannak.
It's not "to be" in general that's droppable, just those third-person present-tense forms.