I agree. The best I could come up with was ''Why did they not go to it (or that)'', but I guess if you think about it, ''Why did they not go with it'' could be loosely interpreted as not 'agreeing' with something. Kinda similar to saying ''are you in it?'' or ''are you with us?'', in terms of asking if someone is agreeing to a plan or idea. That's how I'm going to remember this anyway, maybe it will help someone.