I think he wrote the driver's "smile" to show the indifference, cruelty and darkest side of people in society. Only when 18-year-old travels for the first time, the "I" who has nothing to do with the world will be full of sense of justice and rush up at the most dangerous time, instead of "not hearing" the apple being robbed like a driver and laughing when "I" is beaten.
I think the focus of the novel should be the comparison between the innocence of 18-year-old "I" and the cruelty of people, and the experience and feelings of 18-year-old "I" exploring the world for the first time.