gyda'i gilydd only refers to nhw (they) e.g. Maen nhw gyda'i gilydd (They're together).
gyda'ch gilydd refers to chi (you) e.g. O'ch chi gyda'ch gilydd? (Were you together?).
And gyda'n gilydd refers to ni (we) e.g. Byddwn ni i gyd gyda'n gilydd (We'll all be together).
That's how I would understand it as well.
Aethon nhw i Abertawe gyda'i gilydd "They went to Swansea together" -- the "they" is the nhw and the gyda'i gilydd is just "together", not "they together".
I would like to second the request that "together" be accepted as a translation for gyda'i gilydd.
Incidentally, shortly after I got this question, I got gyda'n gilydd, and there, "together" was accepted (though the "best translation" shown in the sentence discussion was "we together").
For consistency, please accept "together" also for this exercise. Thank you!
We have now got rid of the gilydd expressions that were without a surrounding context - they seemed to be causing some confusion. They are now all shown in sentences to illustrate the patterns more clearly.