It is - as you say - perhaps not the most natural phrasing, but definitely perfectly correct. In many contexts it could by very natural too:
"In the box there is a gun"
"At the end of the street there is a post box"
"Above the window there is a shop sign"
"Above the window there is nothing"
Ничегo is the genitive declension of ничто (nothing, similar to никто which is no one). Нечего is a pronoun that is used to describe the "unpossibility" of something happening. I can't think of a good example dealing with windows specifically, but... В комнате ничего нет, а поэтому мне нечего делать - There's nothing in the room, and therefore nothing for me to do.