Oh, that can be confusing, true. But that means, that "wasza" is used when the noun phrase is singular. There is one fish here, that belongs to two or more people (siblings, let's say). So I will say "wasza ryba" to John and Ann, because it is their fish.
If there were three fish (trzy ryby), I'd say "wasze ryby", as "wasze" is the form for "not masculine-personal plural" (plural for everything apart from 'groups of people with at least one man'). Or "twoje ryby", if I was only talking to John and all the fish were his.