The answer to what they're building is written on Karpathy's twitter - "A kind of Jarvis"