You can only say det stod about objects that are perceived to be standing up. Otherwise there isn't that much difference. fanns emphasizes the 'presence' slightly more and stod would probably be more neutral (at least for objects that are clearly standing), but the difference isn't huge.
It depends on the form of the shelf. If the shelf is seen as enclosing things, it will be in the shelf (i hyllan). Maybe a bookshelf with sides, for instance.
If the shelf is seen just as a flat surface, it will be on the shelf, på hyllan. This could be the case for one single shelf with no sides.
A flaska is used for pretty much any kind of sealable glass container, while a butelj always has the typical cylindrical form of e.g. a bottle of wine. The latter is also far, far less common a word nowadays.