you learned caeredin for edinburgh, the gaelic name is dun edin, dun means also fort in gaelic
Yes, it's another form of "borough" and "-bury" (as in, say, Salisbury).
And the -dydd comes from an old form of Taf, the name of the river which flows through the city past the present castle:
- Mae Caerdydd ar afon Taf - Cardiff is on the river Taff.
A number of now English cities that were fortified places in Roman days have Welsh names that begin with caer-. For example, Caerfaddon (Bath), Caerlŷr (Leicester), Caerwynt (Winchester).
Also the the "car-" at the beginning of "Carlisle" is thought to have come from the Cumbric form of this prefix.