"How do you go to the Olympic stadium?"
According to the official site for the 2020 Tokyo Olympics, they are referring to it as the オリンピックスタジアム. オリンピック会場 is more common vernacular, however.
Nonetheless, your answer should be accepted. Suggest reporting.
オリンピック会場 ('Olympic Stadium') is a proper title or word, which need not be changed.
If, however, you wanted to specify which stadium (会場), you could insert the の for emphasis (The 'Olympic' stadium).