I'm just learning here so someone should verify my answer, but I thing "on" something is "em cima" as opposed to em. So the contracted nesse, nessa, neste etc. would not work in the sense of "on top of something".
I believe estar should be used instead of ser because the letter is not permanently there. Also, em cima is paired with "de" which would mean desse should be used since de + esse = desse. So it'd be "A carta está em cima desse livro"