"There is a window above the door."
Translation:Над дверью — окно.
It sounds unnatural. You can say "Над дверью есть окно", though.
In such sentences you start with the things you already mentioned before (like "the door", it is definite in English) and then move on to the new things (like "a window" which is indefinite in English).
На столе стоит стакан, а в стакане таракан. - There is a glass on the table, and in the glass there is a cockroach.
If you change the word order and say "Окно - над дверью", that would be "The window is above the door".
над, под, перед, за - when indicating position or location are followed by instrumental.
"Надо мною тишина" - Виктор Цой. (there is no stress on the a here, not to confuse it with надо-needs, which has a stress on the a)
"Где кружат чайки за окном" - Вальс в миноре
but if you are indicating movement towards something, then accusative case.
"положите телефон над стол!"
"пойдем за здание" - some random girl
also, if you are indicating a movement coming from something, then the case is genitive.
"монстер из-под кровaти"
But there is no situation where the prepositional case is used with these four prepositions.