"Tard le soir"
when does evening (le soir) end and night (la nuit) begin? Just to get an idea if this sentence could also mean "late evening"
Yes it could. Speaking for myself, evening is from 18h-19h to 22h-23h, but it varies with people, you won't be able to get a rule with that.
I agree with you.
"nuit" is also used to describe when it is dark: "Il fait nuit." = "It is dark."