It's "thing" with the object particle. The way I think of this construction is, "The small child likes the horse-riding thing." Horse-riding is a compound adjective describing the thing. In English, we'd normally replace the horse-riding thing with "horse riding", "riding a horse", or "riding horses".
The bound noun 것 is the object of the verb 좋아하다. So the valid particle here is the appropriate object marker: 을.
Note that you have to use the verb 좋아하다. 좋다 is an adjective and does not accept objects. 좋아하다 is a verb that accepts an object of affection. Hence you use 좋아하다 when speaking about other people's preferences.
In short, this sentence would also be a valid translation.
The sentences "The small child likes to ride horses." and "The small child likes riding horses." largely have the same meaning.
In English, the grammars "like to [VERB]" and "likes VERBING" are largely interchangeable. There are some who might make a distinction between a person's choices and habits in the former and opinion in the latter. This is also true in Korean with the grammars "~기" and "~는 것". The "~기" grammar is closest to the English infinitive "to [VERB]" while "~는 것" is closest to the English gerund "[VERB]ing".
Thus either is grammatically valid: "작은 아이는 말을 타는 것을 좋아합니다." or "작은 아이는 말을 타기를 좋아합니다."