Is there a rule for why "que" = "what" in this instance and "o que" is not used instead?
Because you have a noun after that. It would different if it was like "eu não sei o que fazer". (I dont know what to do) "não sei que/qual carro comprar (i dont know what car to buy)
Is this sentence what someone would say to mean, "We don't know how to react"?