Is there any real rule when the Word "oleh" can be omitted or not?
”Banyak orang” is the agent, so there’s no need for “oleh.”
If you put something between the verb and the agent, “oleh” is needed.
Example: “Labu kami dianggap menjanjikan oleh bank.”
(Our pumpkin was considered promising by the bank.)
In casual conversation, it is optional.