"The waves of the sea"
Somewhat, yes. A few monosyllabic closed-syllable en-nouns get an unintuituve -or. Ros ("rose") is the same in this respect, having the plural rosor.
Apart from what Mark said, you can't use av here. If you want to use a preposition, you could use på: vågorna på havet is a correct phrase.
Basically av is never (or at least very rarely) used to denote ownership in Swedish.
When I did some google-checking before answering, I did see vågorna av havet being used. Can av not be used in the material sense here?
In the English phrase the waves of the sea, "of" can indicate two different senses: "part of" (piece) or "made of" (substance). If I said I listened to the waves of the sea and not I listened to the sea's waves, I'm choosing to be poetic, and doing so implies the substantive meaning of "of". Likewise with the smoke of the fire drifted up my nose versus the fire's smoke drifted up my nose.
No, those are bad translations/too strong influence from English.
If you were to use av with the 'material sense', you should have the indefinite form. vågor av ljus 'waves of light' works for instance.
PS if you just take a quick look at the first hits even in the preview you should see some very serious language errors. "Man Anseendet Vid Vågorna", "Hög kvalitets Stock Foton", "Stor fet fläcken tvättmedel" – sites like that are really polluting the internet :(
Same as if you'd say the sea's the waves or my the book in English. After a word in the genitive or a possessive pronoun, the thing owned needs to be indefinite.