And why is "Indulunk a háztól" wrong?
In "indulunk a háztól", the emphasis is on leaving, whereas in "A háztól indulunk", it is on the house itself.