- should be cheap but if you have kids you know that people can spend a lot of money on toys.
- Should look "tough to break" - must be pretty robust when kid drops it shouldn't break easily.
- Should have text 2 speech built in for all the navigation
- initial set of apps must bring educational value (ABCs, initial math, games that you must count to win etc.).
If you do it right, you can have your own app store that lets parents buy apps therefore allowing you or other developers to develop apps for it (be sure to approve each of them to fit the audience). But you'll need to hide this option since you don't want a kid to even accidentally find it and wondering how to close it :)
I wouldn't start with Android beacuse you don't be able to run it faster on cheap hardware. But maybe linux kernel with ARM support and modified XFCE with touch-screen (capacitive or anything cheaper).
And maybe an accelerometer/compass will be cool since that'll help kids play games easier :) (kids 2/3yrs old don't have firmness in their hand to play games with touch screen)
checkout "Julius", for text to speech on linux.
And yes app store is top a must as this thing wont be anything without an ecosystem.
Also it could be that you would both have a smaller device and a bigger one.
The smaller device is their running around, outside doing stuff and then the other would be their home device.
We don't want to teach our children to sit still and not use their body. So location based games, Treasurehunt etc. could be a way to get them moving around.
It's definitely an under served and under researched field.
Also, please bear in mind that there has been no substantive research to say that any of those education toys actually educate. In fact, it could be argued that they inhibit or at best just occupy time.
However there are people like Seymour Papert who seem to have done some good work in the field. I can recommend his book Mindstorms.