Translation:I go here and there on the weekends.
I believe に would be used to emphasize the weekend.
as you said は is more casual "as for weekends I do a thing" It's done regularly, when you feel like doing it, usually on weekends, when I'm free.
に is more "On the weekend I do this thing" as opposed to any other time I specifically do this thing at this scheduled time. It's a more planned activity. Which for "I go here and there" probably doesn't work well unless you have a designated wandering time. lol