When playing football in "korpen" we kept using the word "pojkar" even when the players were 83 year old... And I could certainly use "kille" regardless of if I was talking about an infant boy, a teenager, an adult or a senior citizen - but never when writing about a boy/man in a formal text. (native)
Yeah, formal text is something else.
I won't question your own native language, but I still think kille is a lot more versatile. You might say jag känner en kille som ... in your 50s, for instance, but I doubt anyone would say pojke där. You could talk about killarna på kontoret, but pojkarna makes it sound a bit like a pilsnerfilm, and so on.