"We are very busy. We only ate lunch at 4pm in the afternoon."

Translation: 我们很忙，下午四点才吃午饭。

[ClassiDuo]

I think in English we'd be more likely to render the second part as "We didn't eat lunch until 4 pm"

[bovelby]

Agreed.