Why isn't it "To jest wasz psa?" Shouldn't dog be in the instrumental since it's the object of jest?
No, that's a simple "This is X" sentence so X (your dog) is in Nominative.
Follow Up Question: Are there other exceptions for when być (and its various conjugations) do not lead to a instrumental object or is it only in "This is X" sentences? For instance, one does use the instrumental in "Y is X" sentences so long as jest is used instead of to, correct? (Also, thank you for always having such speedy and helpful replies, Jellei.)
I hope the link that immery posted will clear all the doubts, but I just wanted to quickly notice that "wasz psa" mixes cases and none of them is Instrumental. Instrumental would be "waszym psem".
How is the word for "dogs" "pies" "psy" and "psa" depending on the case?
Well, cases are quite complicated. The main, Nominative forms are "pies" (singular) and "psy" (plural). The Accusative forms are "psa" (singular) and again "psy" (plural), as not masculine-personal plural has the same form in Accuative and Nominative.
"Wasz" is "your" for plural "you" (more than one person's)
"Twój" is "your" for singular "you" (one person's)
You should be able to, as " to jest" means both this is/that is; and "to są" means these are/those are.
It is possible that "that" may not be added to a list of possible translations, you should report if that happens.