Feynman's lectures of physics are good for layman explanations, however they quickly feel lacking without the maths background, which you can fill in with The Theoretical Minimum. There's actually a video lecture series of the Theoretical Minimum series that you might find more approachable, that is if you haven't heard of it: https://www.youtube.com/view_play_list?p=189C0DCE90CB6D81 (Here for the general relativity playlist: https://www.youtube.com/view_play_list?p=6C8BDEEBA6BDC78D, and here for the full list of playlists: http://theoreticalminimum.com/courses/archive)
You say you haven't heard from anyone who has used it to go from zero to say, general relativity. You might want to look into the adult alumni of the class.
EDIT: You might also find another HN post interesting, it's currently at the top of HN classic: "Learning classical mechanics through Haskell", here: https://news.ycombinator.com/item?id=16453192
(And a direct link to "Classical Mechanics via Scheme", https://mitpress.mit.edu/sites/default/files/titles/content/...)