"There is a small convenience store by the hospital."
It should be, it's not wrong. It's just that usually the when/where comes first in the sentence, and the subject/object goes with the verb. In casual speech it should be no problem to shift them, duo itself does it sometimes.
Your answer would be closer to 'The small convenience store is by the hospital', when the answer they want is 'Near the hospital, there is a small convenience store.' This focus changes the nature of the sentence.