"Your dog is so cute!"
if the correct chinese answer was 你的狗太可爱了, then the English should be "Your dog is too cute!", as 太, used in this context, means too. If so were to be used, then 很 could be used, as well as 好 (你的狗好可爱! or 你的狗很可爱!)
"Your dog is so cute" is a perfectly appropriate translation, as the word 太 as part of the structure 太...了 typically means "so" rather than "too." You're right that 太 can mean "too," but it has several other meanings as well, and the interpretation depends on the context.
Conversely, in 你的狗很可爱, my understanding is that the word 很 would simply mean "is" rather than "very," since it's the default word connecting a noun to its adjective in descriptions.
As others have said, it’s just the expression you use. I’ve found that 了 often comes up in constructions like this that we might put an exclamation mark on, so I remember it that way.
It is "a modal particle intensifying the preceding clause" according to my dictionary
Why wouldn't 很 (hěn) be used in this sentence? Why doesn't Duolingo explain these nuances along the way (assuming it's not an error)?
很 in this context (connecting a noun to an adjective describing it) simply means "is," according to DL's grammar notes. The construct 太...了, while unfortunately not explained yet in the grammar notes, seems to convey the emphasis on degree that's found in the original English sentence, which would be lost if 很 was used.
太 often means "too [much]," but it's also used in this sort of context in a way that's conveniently analogous to the colloquial English use of "too" to mean "so" or "very"; think, "your dog is too cute!"