Yeah. Even in Romanian it says de which is of.
Sometimes, in fact most of the time, we say sticlă de lapte which is precisely a bottle of milk. So if you buy milk or water from the market, it will be sticlă de lapte and sticlă de apă.
But sometimes we say sticlă cu lapte if it is a random bottle that we filled with milk. This would literally translate to a bottle with milk.
I think we mostly use cu lapte/apă (instead of de) if we want to empathize that the bottle is not normally filled with milk/water, but we did so anyway, such that the bottle is not of milk/water but filled with milk/water. That's the reasoning behind our constructs.
Sticla DE lapte might be a reminiscence from the communist period, dominated by scarcity in both food offer and packaging. In that period there was a special type of glass bottle for milk, a special type of glass jar for yogurt, a special type of glass bottle for water, another one for beer etc. They were reused. You were giving an empty bottle of milk plus the money for the milk to receive a bottle of milk with milk. There were no plastic bottles.