Agree. In English, adverb placement is pretty flexible. It could definitely go at the beginning. For a significantly earlier time, one might be more likely to say "Back then it was cold" but the word "back" could be omitted and it probably would be if "then" were referring to a more recent past. E.g. The weather has really changed since last week. Then (last week) it was cold!
"Then it was cold" sounds more like a follow-up to describing something that happened before it was cold, rather than just a simple description of something in the past. "The weather was crazy yesterday. First it was really hot, then it was really cold!"
"What was the weather like around here when you were a kid? "Then it was cold" doesn't work well at all. "It was cold then" is a much better fit. ("Back then" is an even better fit.)