"Daran fehlt es,"
This sentence is confusing. Why do you need 'daran'?
And for the sentence "Ich hatte den Kugelschreiber. Daran fehlt es,". What does this sentence mean? Why do you need 'daran'? Is it wrong to replace it with 'Er' ? Or maybe "..... . Es fehlt,"?
The verb "fehlen" means "to be lacking". "Es" is meant as the current situation, like the "it" in "It is time." So "It (the current situation) is lacking in that." German uses "an" instead of "in" here (prepositions don't translate) and as it is "es/das + an" you get "daran". Order changes for emphasize.
I hope I could be of help.
You mean like: "I don't have a pen, I am lacking one." That would be: "Ich habe keinen Stift, mir fehlt es an einem/mir fehlt einer." The verb "fehlen" has different meanings depending on how you use it. If you put a subject before it like "er" the word tends to get highly emotional. "Oh, my pen I miss you so much." In that sense you can use: "Er fehlt (mir)." But here you want to be impersonal and just say that something is necessary but not available. So you should use the "es fehlt" variation and the only way to connect that to the thing which is lacking is via the preposition "an". So it would be: "es fehlt (mir) an ihm" = "I am lacking in him/it". But that does not sound so well. "Er fehlt." can also work but you have to be careful. Better you add some context here. Like: "Jetzt fehlt er mir zum Schreiben." = "I am missing him for writing now." Or a simple "noch" can work wonders: "Eins fehlt noch." = "One thing is still missing." Or maybe you just use "brauchen" there are not as many hurdles with that word.
I hope I could be of help.
Thanks. Regarding your sentence "es fehlt (mir) an ihm,", why do you need the 'es'?
The "es" comes with the verb here. It points to situation as a whole. It is necessary. It is the same "es" as in "Es regnet." = "It rains." or "Es geht mir gut." = "I am doing well." or "Es steht unentschieden." = "The score is even." Like I said, the "es" gives the verb another meaning or emphasize here, and it acts as the subject. With it you look around and analyse what is still necessary. Without it you look at the indirect object's needs and possibly longings, which can change the meaning completely.
"Dem Haus fehlt es an Farbe." = "The house is lacking in color." (It is white)
"Dem Haus fehlt Farbe." = "The house needs color." (You should really repaint it)
"Es fehlt ihm an seinen Freunden." = "He lacks his friends." (for this task right now)
"Seine Freunde fehlen ihm." = "He misses his friends." (he longs for them).
But you normally don't use this "es fehlt an..." construction with persons: "Es fehlt an dir." sounds off. But just saying: "Du fehlst." will send you off right to the longing/missing meaning. Herbert Grönemeyer once used this sentence in his song "Mensch". If you listen to the song you will know how highly emotional this verb can be. So you would mostly go with "Du wirst gebraucht." for that. Or say it in two sentences: "Einer fehlt noch. Nämlich du."
"Dem Haus fehlt es an Farbe,". Could this mean "There is a house lacking in color,"?
Well, "dem Haus" is definite, so you know which house is meant, probably your house or the house you are inside of. "There is a house lacking in color." would be: "Es gibt/Da ist ein Haus, dem es an Farbe fehlt." with the normal difference: "es gibt" speaking about the general existence of said house and "da ist" pointing to a house over there.
Or did your question have a different reason?
In this context the sentence would be roughly translated to: I had a pen. I'm lacking it (the pen)/ It (the pen) is lacking [now].
To answer the other questions:
Es fehlt an etwas (+ Dativ) is a whole structure, which would be comparable under certain circumstances to there is/there are. "Es" signifies the impersonal character of the whole sentence (just like there in the English example in my opinion). "Es" is the subject of the sentence and it doesn't equals the pen. Thus replacement with "er" is impossible.
Der Kugelschreiber. Es fehlt an ihm. - I'd say it this way. But it's possible to say: Der Kugelschreiber. ER fehlt (that's possible).
Or you could use a type of personal sentence: Mir fehlt der Kugelschreiber.
"Daran" is used here because the verb requires it.
Some German verbs function on their own (intransitive), some require objects (transitive) and some need prepositions in front of said objects to function in a sentence correctly. In this case, "fehlen" doesn't come alone but with a preposition "an" and because the object isn't mentioned in this very sentence (it's assumed that you know what is missing thanks to your general knowledge or thanks to the previous sentence), "daran" is used. You can say that daran functions here instead of the whole prepositional group (which otherwise would consist of a preposition, sometimes an article and a noun).
The word itself is constructed according to a reoccurring pattern (we also have dabei, davon, davor, dafür and others - notice that in the case when a vowel follows "da" we add "r" to make it more pronounceable, thus darüber or daran). It's used in affirmative as well as negative sentences.
There is also a form constructed in a very similar way, which is used in questions. Instead of "da" we use "wo" to form it (wovor, wovon, wobei [careful with this one - it's used in other context as well], wozu). The same rule concerning vowels is applied (woran).