The diary of an inmate
Dagboken til en fange
Dagboken av en fange
Are both valid here or only the first one?
If what you wish to translate is "the friends of my heart", then "hjertevennene mine" would be the better translation. If you really want to stick with the preposition, then I'd go for "av".
I'm not completely sure what you want it to mean though, so please let me know if I misunderstood. :)