I've heard that you REALLY can't use the word miluju except when you're talking about a marriage relationship. It's not a word that is used in normal Czech language at all, and I think native speakers would kinda laugh if you'd use it...
This is obviously figurative, you do not marry Sundays. The sentence is completely fine and it is not dissimilar to the situation in English.
What you have heard was probably only limited to persons, end even there it should be taken with a grain of salt. Sarcastic, ironic and other non-literal usage is certainly possible too even with persons.
Ok, so you can use it about something you love, not only a marriage relationship? Still, isn’t it a quite ‘powerful’ word?