Det doesn't refer to the knife here. Instead, it's the same kind of det used in det regnar ("it rains"). It is there because any Swedish sentence requires a subject. And in constructions like this, det regnar and det finns ("there is"), it's always det.
Maybe it could be compared to French il y a always being il even if you'd say il y a une pomme.
Does that make it a little bit clearer?
We use various prepositions for those. In this case, we'd say eggen på kniven. på is used in many cases, but there are many other cases where other prepositions are used. Like 'a map of Sweden' is en karta över Sverige. If you have to guess, go for på. It's only very rarely av.
Wow this was a while ago, tbh I never really understood the difference until now, was mainly wondering if där är was a possible combination of words, since they are both pronounced almost identically and it would be slightly awkward to say, there's a few English word combinations like that, can't think of any but I know there are. So basically one is existential and the other is more superficial.