"I will leave you with my books" should also be an accepted answer.
Could this be a bequest or part of a will as in 'I will leave my books TO you'? (At times, 'com' is translated in this way.)
No, "i will leave my books to you" would be "(eu) deixarei meus livros para você".
Why isn't it "contigo" instead of "com você"?
Both options are correct.
I would have put that the first time, but the "hint" suggested "I'll drop off," which sounded more natural, so I put that instead...counted wrong. Duo is confusing sometimes.