"The museum is on the right."
Translation:Το μουσείο είναι δεξιά.
In this translation, I used "στο" instead of "στα"; Is "στα" used because "the right" is feminine?
Nope. If it was feminine, it would've been "στη δεξιά", no? ^.^
What I assume you are referring to is the adjective δεξιός, δεξιά, δεξί. Yes, δεξιά is feminine when it's used as an adjective (for exp. στη δεξιά πλευρά - on the right side) , but not in this case, because of that στα.
Στα δεξιά works just like the adverb δεξιά would, implying "προς τα δεξιά". ^.^