Would it be wrong to say, 'alikuwa akianza kujifunza kiswahili'?
I think that's more correct. Past continuous is generally -likuwa + -ki- or -na- .... with another -li- seems strange, like it should possibly be more like the past perfect.