Would "Czy tu muszę być?" be acceptable?
Yes. You would emphasize "tu" more then. You can use "Czy muszę tu być?" as the neutral question.