Translation:He looks at me and asks me if I would like to try.
I think it has something to do with the subordinate clause. "Jeg vil gerne..." is a main clause. However, in a subordinate clause you change the position of gerne and vil: "...om jeg gerne vil...".
As you've said, an inversion takes place in questions: "Vil jeg gerne...?/Vil du gerne...?". But as you can see, it's a different inversion here.
I don't know about the position when linked to an adverb but since Danish grammar is pretty close to German grammar, I guess the position stays the same like in a main close: "Jeg vil gerne løbe langsomt."