I suggest a simple Linux machine running a Gnome DE and learning Racket with Dr Scheme or, perhaps, Smalltalk. Alternatively, if the kid is interested in hardware, go the Arduino route.

I learned BASIC and then Logo when I was 4-8. It made a huge difference to understand iteration and recursion from a young age.

If he is going to be self-directed, something with good learning materials geared towards his age is the key. Those three options should have plenty.