Why use sem instead of tady?
sem is a direction (hither), tady is a location
They are not going to be to go(come) here- could be accepted?
They are not going to go, to come, to be going, to be coming - all of these are accepted.
You can't say "to be to go/come".