"I like sitting by the window."
Translation:Ich sitze gerne am Fenster.
I'm just a learner, but I would say the difference is the same as in English between "at" and "by".
For example, look at the different meanings of "Stay by me" and "Stay at my home".
That's how it is with prepositions, you just have to learn them from scratch with every new language.
Also, in this context "an Fenster sitzen" (at) means that you are literally at the window, looking out at the beautiful world around. On the other hand, "bei Fenster sitzen" mean that you are sitting somewhere by the window, like next to it or somwhere near it, and this expression defines just your location, not an activity.
Just like sitting at the table ("an") means eating a meal, and sitting by ("bei") the table is just being somewhere near the table. At least that's how I understand it :)
"An" means just 'at'. "Am" is contraction of "an dem" and means 'at the' (used for masculine and neuter dative).
The word order feels off, although I cannot really explain why.
Perhaps because an adverb of manner needs to be in front of an adverbial of place? But I can come up with a counter-example almost instantly, so I'm not sure.
Is "Ich mag an dem Fenster sitzen" really wrong? Or just not added as one of possible answers?