Literally "è ora di cena" is "it is the hour of dinner." If you use "è ora della cena" it means "it is the hour of THE dinner." Possibly this sounds as awkward in Italian as it does in English. Hope this helps :)
I just read the example: Il vestito DA sera,
and comments explained:
DI - is From;
Da - is Of;
.... and of course, the Dress can not be OF Evening (DI)
.... Now, How can the Dinner be OF Time?! or Time OF the Dinner?!