It is incorrect in American English, which is what it seems Duo is based on. But it's not THAT outlandish or hard to understand. You're not helping our (American) reputation, Donald... It's pretty instantly recognizable that "Have breakfast at mine" means "...at my place" and definitely not "at the coal mine"
"by me" isn't great English here. I (US English) would take that to mean that they have breakfast somewhere near my place but not at my house and we don't have breakfast together.
"With me" would be perfectly fine English, but I don't know if it works with the Russian. I wonder if that wouldn't look more like "Они с мной любят завтракать", but I could have that totally wrong.
"со мной" but otherwise exactly right. "they like to have breakfast by me" I would take as "somewhere in my vicinity" or "next to me" or similar. In Russian it would me more "Они любят завтракать рядом со мной" or "Они любят завтракать около меня" - just as odd-sounding as in English.
That would sound awkward in English. "Site" refers to a geographic place, yes; but in more of a professional or scholarly capacity, especially when there's no actual building present - e.g., an archaeological dig, an empty lot to be developed, or in contrast to another place in a similar situation, as in "this site" vs. "that site". "My place" is generically understood to be where someone resides, although one could also specify "my house" or "my apartment/flat". Or in this sentence if a restaurant owner is speaking he could be referring to his/her establishment.
It seems to me that любят means more than mildly like (нравится). With this in mind, They love to have breakfast at my place would be the best translation to American spoken English. Literally when referring to something other than a person, love means "like very much" in American spoken English. In formal written English, the best translation is "They very much like to have breakfast at my place".