"Thank you very much!"
There are three correct word orders: 1- Nagyon köszönöm = thank you very much 2- Köszönöm szépen = thank you kindly (literally : I thank you nicely) 3- Nagyon szépen köszönöm (I thank you very kindly / 'very nicely')
So, in the third sentence 'nagyon' refers to 'szépen' , not to 'köszönöm'.
'nagyon köszönöm szépen' is incorrect - or at least it would sound very strange in everyday Hungarian conversation. Stick with 1-3 above to be safe. :-)
From a word order perspective, this is fine. More literally nagyon is 'very' (adverb of nagy, 'big'), making nagyon köszönöm roughly translate to "Big thanks from me"; and szépen is more along the lines of 'dearly' (adverb of szép, 'pretty, nice'), making the construction say "I thank you nicely."
While writing this I realise how exotic that language is indeed. :D
Yes, right. Köszönöm szépen seems to be the odd-one-out since szépen remains unfocused. Nagyon clearly belongs to szépen and not the word and they take the focus in Nagyon szépen köszönöm. In addition, nagyon köszönöm szépen would even be weird. Why add two relatively similar adverbs to the same verb, one focused and an extra onfocused? Wouldn't make much sense.
I do think idiomatic expressions have a bit more leeway in terms of "Why is that so?" There's almost never a good reason for one version above the other, just that one got popular and the other sound strange now.
I'm leaning on MannaSz's reply to you above and what I learned. "Nagyon köszönöm szépen" somehow sounds off. But maybe it's a regional thing?