Since you seem sincere, I’d like to mention that neural networks (as of now) are a complete clusterfuck for a problem with as much structure as you’re describing. There are no known ways to impose the relevant constraints/invariant on neural network behavior — and stop them from producing junk — leave alone doing something useful. That Karpathy article is pure hype with very little substance (like most commentary on neural networks). I like the vision, but it might take a minimum of twenty to fifty years to get there.

If you consider yourself a world-leading expert on neural networks and have some secret sauce in mind, by all means, good luck... otherwise it sounds like a fools errand.