I want to understand more about what's actually happening. What are the big concepts that, once you get them, make everything click in a more interesting way? The stuff that made you go "oh, THAT'S what's going on."
But your question is so vague that there are a lot of answers.
There are a lot of stuff that can be considered big concepts : basic electronic, networking, OS, peripheral communication, compilers, language, algorithms, assembly/procedural/object/functional programming, and the whole world of "AI".
What do you want to know?
- why you shouldn't call .where() a million times in linq?
- how internal statistics tables are managed in Sql server to produce the execution plan, how to watch the execution plan?
- big endian vs little endian?
- how to use telnet to get an HTML page from a web server?
- why everyone thinks that their favourite language is the best?
- why is the first char in a string in pascal at [1] when its at [0] in a lot of other languages?
Once you get how data moves, where state lives, and what happens when things break, a lot of the "magic" disappears. Learning basic debugging and reading stack traces also compounds forever, even with AI.
One thing that helped me: think a bit like a PM. Spend time crafting good prompts (use chatgpt for generating prompts and feed it to your AI coder ;) ) for ChatGPT/Claude/Cursor, ask it to generate docs and explain what it just did. Over time, patterns emerge and things feel less mysterious.
LLMs write code well, but they don't give you mental models. That part you still have to build.
Its also why you'll hear many engineers opine over functional programming.
Most start by thinking about 1. What does a system do for me?
Then onto 1. What actions does a system do? 2. Where should files and folders live? 3. Where should services live?
and stop there.
But the true key is when you just think about how data moves through your system.
Then you learn to simplify.
No classes, no hidden state, just simple functions with an input and an output and nothing else.
Long story short: Think about your system in data, and learn how to keep your code super simple to help with that thinking
One thing that helped us: externalize the structure that experienced developers carry in their heads. Things like test driven development or wheel-and-spoke based file size limitations etc. are the distilled judgment of decades of software engineering. But if you've never written code traditionally, you don't know they exist.
We formalized these into enforced workflows. What I found pretty exciting about it, from an educational tool standpoint, is that the side effect was that new team members and vibe coders working within those constraints started absorbing the patterns themselves. They learn why tests matter because the system won't let them skip them and learn why file size matters because the system blocks them and forces decomposition etc.
* SICP - Structure and Interpretation of Computer programs
* AOA - Art of Assembly language programming
* Hello Python - Think like a computer scientist
* The C Programming language - Kernigham and Richie
* Pick one more favourite language and a book in it - Rust, Golang, Scala, Java, Ruby, Python, Typescript etc
I made a /teach-me slash-command. Everytime I run into something I want to know about I run it. It launches a background subagent (forking context) that check (and updates) my ‘learning-profile.md’. I can pass additional requests, but otherwise it just deduces from context.
It then creates an explainer/lesson. And sends it to my kindle (using a custom created mcp) - so I can read it offline. Not breaking my flow.
I’ve never learned so much so quickly, about everything. It’s awesome.
I use the same mcp from phone apps. Everytime I want to learn something I just ask.
Also: learn about the GIT concepts (not commands). You’ll learn much quicker when you can mess up and retry ;)
I am not a developer. Could you explain the process in a bit more detail? I want to build the same command. I only have the $20 subscription for Claude. Do I need to create a project and create a learning profile.md file and feed what I already know? And then what do I do?
Sorry if that's too many questions.
Rather than viewing AI as forbidden or universally permitted, consider this progression:
1. Foundation Phase (Avoid generation, embrace explanation)
When learning a new library (e.g., your first RDKit script or lmfit model), do not ask the AI to write the code.
Instead, write your own attempt, then use AI to:
• Explain error tracebacks in plain language
• Compare your approach to idiomatic patterns
• Suggest documentation sections you may have missed
2. Apprenticeship Phase (Pair programming)
Once you can write working but inelegant code, use AI as a collaborative reviewer:
• Refactor working scripts for readability
• Vectorize slow loops you have already prototyped
• Generate unit tests for functions you have written
3. Independence Phase (Managed delegation)
When you have the skill to write the code yourself but choose to delegate to save time, you are essentially trading the effort of writing for the effort of auditing. Because your prompts are condensed summaries of intent rather than literal instructions, the LLM must fill the "ambiguity gap" with educated guesses.
Delegation only works if you are skilled enough to recognise when those guesses miss the mark; if your words were precise enough to never be misunderstood, they would already be code. Coding without oversight is dangerous and deeply incompetent behaviour in professional environments.
Examples of use-cases are:
• Generate boilerplate for familiar patterns, then audit line-by-line
• Prototype alternative algorithms you already understand conceptually
• Document code you have written (reverse the typical workflow)
There's different books or approaches to learn more depending on your needs.
It's usually useful to tailor things to whatever you already produce for others or consume for yourself because then you get the most of your feedback loop.
Can you share more about what you've tried so far, with what tools, or what feels the most fun or useful ?
Depending on your answers one might recommend things like
- https://automatetheboringstuff.com/
- land of lisp (comics and games with functional programming to intentionally separate you from vibe coding traditional tools)
- web related dev with different viz or presentation tools
useful concepts regardless will be, in my mind, unit testing and how to approach problems and breaking them down but it all depends on where you want to go.
Here are some practical suggestions based on what I've seen with my students. To summarize: AI programming de-emphasizes the specifics of languages and frameworks but rewards having a good knowledge of systems and a careful, structured development process.
1. Do learn some by-hand programming in a standard language like Python. Even if you aren't looking at the AI-generated code, this will teach you the building blocks of software design - things like functions, classes, files, and data types - which will help you design more complex applications. The ability to look at a program and reason about what it's doing is a key skill.
2. The terminal environment if you aren't already using it. This will unlock additional levels of control and help you understand how agents use tools.
3. Architecture of the applications you're creating. If you're making web apps, for example, learn about the front-end and back-end, how they exchange information, how a back-end database works, etc. The key is not the low-level details of those things, but how an application is divided into parts that exchange data with each other. This knowledge helps you move from a general concept for an application to an actual design.
4. Related to that point: the concept of encapsulation in software design. This is the idea that each part of your system should be self-contained and exchange information with other parts through a well-defined interface. If a component is encapsulated, you can change its internal details without messing with the rest of the system. This is important for AI, because it allows you to carefully control the targets of your generations.
5. Specs-driven development. This is the evolution of vibe coding and is the main approach I teach now. Chat about the problem, then develop a detailed spec that describes the desired behavior. Refine that into a system design and detailed step-by-step task list with tests for each step. Working with the AI to compare design options is a great learning tool.
6. You don't need to learn much about algorithms unless you want to. Models are very strong at choosing and implementing all of the standard algorithms. Let these emerge naturally as you work on interesting problems.
The Programming with AI class is here:
https://github.com/dansmyers/ProgrammingWithAI
We're building it out as the semester goes on. Right now, it's mostly the in-class practice activities we're working through each day. This is the first time I've taught this version of the class so it's pretty experimental.
Here's the complete version of my intro class with notes, labs, and projects:
And don't you dare to ever open a pull request with AI generated code in any free and open source project.
I am never going to be a software engineer either but I took a CS101 class and then data structures and algorithms.
Going from zero to passing those classes was the most useful combination of things I ever learned.
I am so glad I got to do this way before LLMs because I think I am absolutely the type of person who would have used LLMs to cheat and learn nothing from the class.
I think at least those concepts are vastly worth struggling with without the help of LLMs.
Raw dog code on small projects while avoiding using AI, except to explain things when you get stuck.
Learn theory. Learn algorithms. Learn data structures. Learn about secure coding. All of these will help inform you, even if you stick with agentic coding.
Go as deep or as shallow as you want to.
If you want to go deep, read some of the classics (SICP, C book, Pragmatic Programmer, etc), but don't beat yourself up if you don't find them interesting. Most programmers haven't read those books either.
Teachyourselfcs.com is good too
Don't you mind about the future. Don't you try to think ahead. Save tomorrow for tomorrow. Think about today instead.
I could give you facts and figures. Even give you plans and forecasts. Even tell you where I'm going.
as always: imho (!)
recognize that software-engineering is not about writing / vibing code but to solve (!) problems.
nobody cares if the code which solves a problem is generated / copied / written ... as long as it was legally obtained ... ;))
anyway: code is liability, every line of code which was not written to solve a problem keeps future maintenance-costs low(er) ...
additionally especially for non-trivial problem-solutions - read: projects -, its essential to have maintainable code. which means, code that is ...
* easy to understand ~ new developers
* easy to extend ~ new features
* easy to sustain ~ update dependencies, update the underlying runtime-environment etc.
especially if it solves a complex problem for a company, the code may be used for years or even decades =?> keep that in mind!
just my 0.02€
Bottom up and top down.
Bottom up would roughly be 1. Picking a simple introduction to programming textbook ideally Python 2. Work through a building a transformer LLM in python 3. Move to training it on a corpus
You're not mastering each step. Reading the python book and doing some exercises is fine.
The top down: This 3Blue1Brown playlist will have you covered https://youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_...
Either way you want to meet in the middle. There is still a lot in the middle that isn't clear so don't try and work from the middle out!
What this means is coding becomes accessible to those looking to apply something like python to solved problems, but it very much remains inaccessible to those looking to solve truly novel problems they have the skill to solve in their domain, but lack the coding skills to describe.
As a simple example, claude code is easily among the most competent coding interfaces I know of right now. However, if I give it a toy problem I've been toying with as a hobby project, and it breaks so badly it starts hallucinating that it is chatgpt.
``` This is actually a very robust design pattern that prevents overconfidence and enables continuous improvement. The [...lots of rambling...] correctly.
ChatGPT
Apologies, but I don't have the ability to run code or access files in a traditional sense. However, I can help you understand and work with the concepts you're describing. Let me
provide a more focused analysis:/insights doesn't help of course, it simply recommends I clear context on those situations and try again, but naturally it has the same problems. This isn't isolated, unless I give it simple tasks, it fails. The easy tasks it excels at though, it has handled a broad variety of tasks to a high degree of satisfaction, but it is a long shot away from replacing just writing code.
Bottom line, LLM's give coding a GUI, but like a GUI, is restricted and buggy.
They'll still need to pick up the fundamentals of the programming — that part isn't optional yet. And getting to that level as a non-programmer takes real effort. But if the interest is there, it's far from impossible. In fact, I'd argue someone with genuine passion and domain expertise might have better odds than an average developer just going through the motions.
Think of it like image generation AI. You can make acceptable if sloppy art with it, using styles that exist. However, you cannot create a new style. You cannot create pictures of things that are truly novel, to do that you have to pick up the brush yourself.
coding with llms is the exact same thing. It can give you copies of what exists, and sometimes reasonable interpolations/mashups, but i have not seen a single succesful example of extrapolation. Not one. You simply leave the learned manifold and everything gets chaotic like in the example i provided.
If AI can make what you want, then the thing you made is not as novel as you thought. You're repurpising solved problems. Still useful, still interesting, just not as ground breaking as the bot will try and tell you.
About your question, it is hard to give one. I don’t think there is one big thing that makes everything click, and if there is one, it is probably different for different people. But I can give some advices.
1. You can ask the AI itself to explain how the code works. In my experience they are usually fine at that. You can probably tailor the explanation to your technical level, so that’s neat.
2. Stick to one language. I don’t know which language your AI is using for your tools (probably Python if they run on your machine or JavaScript if they work on a browser). Learning what you are using is the first step.
3. Once you know that, you can use the AI and some online guides to learn the very basics of the language. Maybe ask the AI for very simple toy tools (e.g., a web page where you write a phrase, click a button, and it will show that phrase with the words in reverse order) and try to understand what the code does. You may still ask the AI about the lines you don’t understand.
4. It is a potentially long journey. Go as far as you like. After these first steps, you will likely have more specific questions. That’s good. :)
That comes from years of 70 hour weeks of hand coding!
For the former, I think some hand coding will get the best results.
A large part of "thinking like a programmer" is thinking about the sort of invisible structures that code will produce in a computer's memory when it runs - which is not the same as what's literally written in the code file. IMO, the best way to get a feel for that is to write code yourself, run it, examine the results, etc.
But writing code can be a rather dry activity if you don't have motivation for what to write. The usual vibe coded projects are probably too complex to hand code without experience. I think two good starting points could be Code Katas [1] or the Pico8 game engine [2] (and of course any good "intro to Python" or "intro to JavaScript" tutorial on the web)
--
For the latter, you could maybe start with the history of generative text models (like markov chains) and neural networks. Both technologies are several decades old and sort of form the very basic building blocks that made development of LLMs possible.
In particular, neural networks made use of the gradient descent and backpropagation algorithms that are still the basis of most LLM training processes - even if they might now run for months and take exabytes of training data.
Then there is of course the "Attention is all you need" paper that introduced the Transformer architecture - the kind of latest-generation neural network that underpins most LLMs - and basically started the whole LLM revolution. (It's also the "T" in GPT)
(For historical interest, there is also the whole theory of formal languages/grammars and automata - which was the previous attempt to understand human language with computers. That theory gave us a lot of things - such as the Turing Machine and the entire theory of how to build and read a programming language. What it didn't give us though was a way to process human language...)
--
As for the specific question of how an LLM can take this request in human language and generate that code from it that happens to fullfill all the requirements - that's an active area of research and I honestly doubt that right now anyone understands that in all entirety.
Through I believe Anthropic and LessWrong had some interesting blog posts about this topics where they tried to understand the functions of certain specific neurons in an LLM. See also "Golden Gate Claude".
Use AI for building still. But have non AI sessions for learning. Good luck!