larsfaye

>And the fact that I'm a generalist is really playing to my strength for the first time

I was just speaking with my wife about this a few days ago, that I am actually happy that I didn't specialize in one specific discipline, or rather spend my time around learning a little bit of everything, which now gives me the foothold into any direction that I want to go.

> It's time to reskill for AI.

I'm just curious, what do you think that looks like for you and for others? AI is not so much a skill as it is an interface, and one that is quite possibly one of the easiest to use that we've ever created. It is an interface for information, essentially. It's the skills that you come to the table with that determine the results that you're going to get back, and whether they're going to be quality or slop.

It sounds like what you're describing is similar to this new course that Theo Browne (ugh) and Kent C Dodds (double ugh) are launching called "mega.dev", and it appears that it's essentially just talking about the systems and tooling that's going to go into utilizing agents, and code is no longer even a topic.

> The long term impact of AI on society is exclusively a net positive if utilised in the correct way.

Hi, I'm Earth, have we met?

This is EXACTLY how I am working with these tools, you summarized everything about why I use this approach perfectly. Zed + OpenCode + Mimo/GLM/Kimi models are incredible. I haven't touched a "frontier model" for months.

I really treat them as "delegation layer". They're systems that sit right alongside your stack, and at this point, at every level in it, like a connective tissue that intersects at all points. I can delegate a spec, a function, a file, a whole feature set, an app, a debugging loop, etc.. My day-to-day isn't all that different, it's just like having the "Ship's Computer" that I can ping anytime I need something. I'm enjoying the benefits that a smart data processing pattern interpolator provides, but I'm not giving up any personal control or agency. My goal isn't to go faster, nor is it to avoid coding, because coding isn't drudgery or below us just because we have new code generation tools.

>I am glad that I live far away from the effects

Are you not on planet earth? Unless you're on Mars, you're not far away from any effects.

I originally found the paper through this video:

https://www.youtube.com/watch?v=HTUh0OO6Kmo

The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!

It's basically laid out in the article: the reason that you are able to gain so much from LLMs is because of your decades of experience that predated their existence. Ability begets ability, but if that ability doesn't take root, these more advanced toolsets don't yield the same outcome (and are likely to do harm).

To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.

I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.

I posted this elsewhere, but that story has a lot more nuance.

Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.

Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).

AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.

May I ask how many years experience you have?

The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.

Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.

Plato wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.

Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.

AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.

That's because the institutional knowledge was passed along through the act of coding. Skills that I learned in Assembly and FORTRAN are applicable to even modern programming languages...but not to "natural language."

Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.

Literally all those skills are the result of decades of friction, challenge, and deep domain knowledge...all of which are circumvented when LLMs are utilized early in the process.

CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.

Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.

Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.

https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...

Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.

https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...

Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...

> I don't know how you teach that without coding

That's the neat part: you don't!

Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:

https://www.youtube.com/watch?v=RxxxGkFIUJ0&t=1356s

"How lucky I am to have something that makes saying goodbye so hard" - Winnie the Pooh

The extended title is more like:

"AI Coding will Prevent Expertise if We Continue To Focus on Code Generation rather than Socratic and Educational Workflow Design"

...but I had to compress it for brevity! The points and studies I bring up in the essay support what you're saying.

Problem is: I'm seeing very little movement in that direction, but a whole lot around the abdication of responsibility when it comes to AI coding in general.

Yeah...I know I should be massively impressed, and I am to a degree, but isn't this what we should expect? LLMs can pick up on patterns that no human can see. In that sense, they really are a type of "search engine", but I use that term loosely. Tao is using them as a way to sift and sort theories and information that the model has a plethora of training data on. It's like being able to converse with the codex of humanity, and extract data via an algorithm that is highly adept at sorting through it.

Correctamundo. That's EXACTLY what I just wrote about. And how we'll create the next generation of people who know how to evaluate responses when we're creating a generation of people who are increasingly reliant on LLMs to do the work makes for a strange paradox.

https://larsfaye.com/articles/ai-coding-will-prevent-experti...

Uh, he's saying it because the kid is overly confident and acting as if he knows Williams' life because he saw a painting and drew a bunch of conclusions, so he was providing a life lesson to realize that experience provides a level of understanding that book knowledge will never be able to capture. And then he even re-affirms this to Will and emphasizes that he can't possibly know what makes him tick because he read Oliver Twist. Your tangent seems entirely unrelated, IMO.

I actually agree with you; I definitely feel I'm able to use LLMs to learn and explore concepts, but I've always been a self-taught and highly motivated person in the first place. Everything technical that I know, I know because I put in the work to learn it on my own. That's why I wish these tools were being advertised as models that help you do BETTER work, not MORE work. They're using them as excuses to lay off swaths of workers, instead of allowing them to uplift people's skillsets. And, of course, it invited a whole mass of individuals who use them to artificially elevate their perceived skills.

Oh ho ho....looks like I'll need to update my essay! The evidence is mounting higher and higher.

https://larsfaye.com/articles/agentic-coding-is-a-trap

This is exactly how I feel. I've begun referring to them as a "Delegation Layer", which is a new addition to the stack.

Not only can it not describe its reasoning, it can't "remember" if you ask it later; it can only observe what is. Nor can it be consistent; I've had it shift reasoning numerous times as the questioning continues, only to come full circle to its original statement while it apologizes profusely for being misleading.

The model will always be completing the story you start with it. There's no opinion to uncover because there's no experience that occurred. It's impossible to know where your influence ends and the model's factual basis begins.

Yes—this is the part that nobody is telling you. That's the real shift that is happening.

The "don't look at the code" movement is a fad that we simply have to go through to prove that the real problems were never obvious, but lie in the subtle nuances and interplay between all the moving parts of what makes software function in the first place. We have to abuse the tool before we learn to properly use the tool.

I do agree with this. I was able to get exactly what I needed even out of GPT3.5. If you put enough parameters and examples, along with a real solid system prompt and (if you can) proper temperature and topK/topP, there's no reason they can't basically function like "smart typing assistants". The issue is that its a sliding scale of ambiguity to chaos. The more ambiguity, the more the LLM fills that in. And it can be very difficult and time consuming to know when you're being ambiguous or not (you don't know what you don't know, OR, you can't track what you aren't tracking).

Depending on the task, it can sometimes be just as arduous to produce enough guidance and guardrails to get the LLM to output exactly what you need that you can trust without issue or extensive review than it is to write it yourself and use the LLM just for ad-hoc generation. It's a constant balance and an endless amount of micro-decisions, honestly, but it's pretty essential to stay engaged and not YOLO with agents the way so many are. Most of my interactions with models these days are done in pseudo-code.