They're often interchangeable, but there are some differences:
"Stykke" can be used as a measure of length:
"It's only a little bit further to go."
"Det er bare [litt/et lite stykke] lengre å gå.
"Stykke" is also used when talking about the kind of pieces which are intentionally sliced up and passed out, not expected to ever form a whole again, such as a piece of cake or pizza.
"Bit" is both used in a general sense, and to refer to a "bite of food". For example, you may ask someone for "en bit" of whatever they're eating if you just want a taste of it.
For game pieces we use "(spille)brikker"