Actually, "δεξιά" functions as an adverb of place (where?), so that's why it takes the neuter plural article. Some other examples:
Προς τα μπροστά/Προς τα εμπρός=Forward(s)
Προς τα πίσω=Backward(s)
Partly cross-posted with StellaNtal's comment, which I hadn't noticed
In another version of this sentence, translating to Greek, not selecting words, the correct answer was Το γραφείο είναι στην δεξιά πλευρά, and this is considered wrong. The explanation is that δεξιά is an adjective and needs a noun πλευρά=side, and the ά ending is because πλευρά is feminine.