Yes it would be. As written, I think the sentence puts more emphasis on "the teacher" (although I'm unsure of that), but either way you're good to go. The basic meaning is the same in either case, and both are grammatically correct. Actually, I think your proposed alternative would be the more common way to say this (unless you just really had to stress that "the teacher" was the person you were eating with).
"mit dem Lehrer" ist eine Präpositionalphrase, die hier wie ein Adverb benutzt wird.
Aber in der ersten Position, vor dem Verb, können nicht nur Adverbien stehen ("Heute esse ich Fisch", "Langsam trank er sein Glas aus") sondern auch Adjektive ("Gemütlich habt ihr es hier", "Grün ist das Land, rot ist die Kant, weiß ist der Sand, das sind die Farben von Helgoland"), Dativobjekte ("Ihm wurde schwarz vor Augen", "Dir helfe ich gleich"), Nebensätze ("Wenn du nach Hause kommst, werde ich noch wach sein"), .....
The case system in German uses inflection to express syntactic relationships in a sentence, while English does that mostly with word order. That's why the word order can be usually be moved around like this, which is done for the sake of emphasis. For example in English you can't change the word order in the sentence "he likes the dog" without changing the meaning, but in German the same thing could be expressed in two different word orders: "er mag den Hund" and "den Hund mag er". This is because "der" being conjugated to the accusative case form "den" is what retains the syntactic relationship between the object and the subject, while the same isn't possible with English. An exception to this are all personal pronouns apart from "you". English used to have the same system but discarded it hundreds of years ago, however accusative case is still present in the words "me", "us", "her", "him" and "them".
By the way, one thing that can't be moved around is the conjugated verb. It must always take second position in a clause. The exception to this is when it's in a question without an interrogative adverb, in which case it must take first position. If the verb or something else takes first position, the subject must come after the verb.