I can't hear the audio at the moment, but what you write makes sense. If you were to slow the pronunciation down, the j would make a distinct "y" sound, as in English "yellow". However, as soon as you speed up and start having other words in the phrase as well, that sound tends to become silent. Hence, the word is usually pronounced rather like földe.
You have to listen very carefully, because the "j" is being very very subtly pronounced. You are right that it is not emphasized, but it is not quite silent. This is a good example of how Swedish contains many very subtle nuances that can be easily missed by non-native speakers, and why you need a lot of real life practice speaking with a native to really master the language (something I have not yet done!)
Wondering why this isn't följde efter. In a lot of fiction I've been reading I see "följe efter" a lot when the characters go with each other somewhere... ... Bilen svänger in på en liten grusväg... Tom och Lena följer efter.
There's obviously a subtlety I'm missing here.