Couldn't this also mean "Which men were in Norvos"?
I also have the same question. Since the past perfect words for both "sagon" and "jagon" are identical, I think the answer is yes, and the real meaning has to be derived from context.
Side note: "Which men were from Norvos?" would translate to "Spari vali hen Norvot istis?".