They're often interchangeable, but there are some differences:
"Stykke" can be used as a measure of length:
"It's only a little bit further to go."
"Det er bare [litt/et lite stykke] lengre å gå.
"Stykke" is also used when talking about the kind of pieces which are intentionally sliced up and passed out, not expected to ever form a whole again, such as a piece of cake or pizza.
"Bit" is both used in a general sense, and to refer to a "bite of food". For example, you may ask someone for "en bit" of whatever they're eating if you just want a taste of it.
For game pieces we use "(spille)brikker"
Want to chime in with @Volker_Grzimek, @Ihor610881, @Nick1174, I have repeated way too many times and lost count. Duolingo never accepted my second piece. FWIW, "Stykke for stykke" has been drilled into my memory. I am considering to have ""Stykke for stykke" on my tombstone.