This sounds wrong to me in English; do we use 'both' with a negative? I would say 'I don't remember either employee' even though this changes the formulation completely.
"I don't remember both workers" has been considered wrong. Is there the difference between worker and employee??