We have either B or P, but in Korean there're 3 sounds, ㅂ ㅃ ㅍ. We studied phonology a bit in elementary and B and P are very similar, P being the "silent" version of B. Try whispering both B and P and you'll see there's not much of a difference. There're a lot of pairs like this (D and T are ㄷ ㄸ ㅌ).
I prefer using the terms "voiced"/"unvoiced" over "silent" but yes. But there are other aspects to it in Korean--especially aspiration. My understanding is ㅍ is the only "p/b" letter that should be aspirated in Korean (similar to ㅌ and ㅋ and ㅊ). Native English speakers tend to aspirate voiceless stops (p, t, k) when they begin stressed syllables, but not when they begin unstressed syllables. For example, "paper" has first an aspirated p, then an unaspirated p.
The other distinction to be found is in "tenseness" where double consonants are more "tense" or "emphasized" but I haven't figured out how to explain that in English yet.
Wiktionary states that both words share the same pronunciation, with the IPA for both words being [p͈a̠ŋ]
But if you check Forvo, it sounds like 빵 is more "b" like and 방 is more "p" like