Hacker News Reader

Comment by kolinko | Hacker News Reader

I'd say that especially because assistants are not good enough yet, it makes sense to allow students to use them. LLMs will produce output that is messed up, students will spend time figuring out how to fix that and learn the language grammar this way.

I recently learned Swift & Metal this way - knowing Python and other languages, I kept pasting my pseudo-code, asking GPT to convert it into Metal/Swift, and then iterating on that. Took me a week or two to get the language without reaching for any sort of manual/tutorial. Speaking from experience - it would take me 2-4x as much time to learn the same thing otherwise.

If GPT was any better, I'd have no need to fix the code, and I wouldn't learn as much.

View on HN

I want to emphasize that I'm not really a luddite here. I pay for ChatGPT (well, Kagi Professional), I use it daily, and I do think it's a very valuable learning tool. I even encouraged my students to use it outside of class if they needed help with any of the theory.

I use ChatGPT all the time for learning new stuff, but a) I already know how to program well enough to teach a class on it, and b) it's a supplement to doing stuff on my own, without AI help. I don't feel I learn that much from just copying and pasting.

Totally agree with your process though; correcting and arguing with ChatGPT is an insanely good way to learn stuff. It's really helped me get better at TLA+ and Isabelle proofs.

Just FYI, Kagi's FastGPT is not remotely comparable to GPT4 in quality.

I don't use FastGPT, I just Kagi Professional with the GPT-4 model.

Ah ok. I'm on Kagi Family which doesn't have access to that, didn't realize it was an option for Pro!

Yeah, they don't advertise it very well. It was actually a pretty good value add for me; I was already paying $10 for Kagi search, and $20 for OpenAI, so the $25 for both actually saved me $5/month.

I think you can upgrade to Ultimate at a per-account level for another $15/month.

EDIT: Accidentally wrote "pro" when I meant Ultimate.

Probably still not worth it for me; I use gpt4 fairly heavily via API and still my API charges only come to about $5-7 a month.

Did you write your own tool to query ChatGPT via API, or what are you using?

I use typingmind.

Do you mean Kagi Ultimate? I was under the impression that the Pro plan didn't have access to GPT4.

Yep, I mistyped...I meant ultimate. Edited the comment.

You can upgrade any family member to Kagi Ultimate plan for $15/mo and it will give you unlimited access to GPT-4, Claude 3 Opus, Gemini 1.5 Pro and other state of the models.

> I recently learned Swift & Metal this way - knowing Python and other languages, I kept pasting my pseudo-code, asking GPT to convert it into Metal/Swift, and then iterating on that.

Significant here is that you knew other languages already. I've had the same experience with GitHub Copilot, but I'm cautious about recommending it to new learners who don't yet know the fundamentals.

All indications I've seen indicate that it's easy for people who already know the foundations of programming to use AI tools to learn new tech, and that can actually be the most effective way for them, but that it doesn't work nearly as well for people who don't have knowledge of other languages and frameworks to lean on.

> and that can actually be the most effective way for them

A lot of it is just a general lack of learning material for that demographic. For most languages, you get to choose between an introductory text that assumes you've never even heard of a "pointer" or an advanced text that is meant as a reference for an experienced developer of the language.

There's not really a lot of stuff out there to teach someone who already knows another language, who just wants to know the syntax and common idioms in that language. For example, a C programmer can easily pick up Python, learn some syntax, and go nuts -- but they just won't know about stuff like list comprehensions or dataclasses unless someone points it out. They'll write Python code that reads like C, not like Python.

It's similar with maths - calculators are invaluable, but being actually good at advanced maths just NEEDS you to grind through basic calculations so you adopt those basic "pattern recognition" skills on how numbers work, can be broken down and swappe out.

> ... students will spend time figuring out how to fix that and learn the language grammar this way.

That is a cute thought. Instead, they do submit the LLM generated code as is, messed up or not, and still expect to get full grade.

do they really expect a good grade from a messed up code? How many of them?

Yup. I gave a lab homework to an SE class of 42. After reading through at least 12 LLM generated, similar looking, and wrong code submissions, I've cancelled the whole lab assignment. And students complained in class evaluations about it.

An alternative is to keep the lab assignment but grade them on a final exam. Then it is in their interest to do the lab assignment by themselves rather than with an LLM.

>LLMs will produce output that is messed up, students will spend time figuring out how to fix that and learn the language grammar this way.

I’ve never met anyone who prefers debugging over writing fresh code. Did LLMs just automate away the fun part and leave us with the drudgery? That sounds like a horrible way to learn. All work and no play…

It also would mainly teach how to get good at spotting those little mistakes. But without context from writing code, it seems like it would be harder to pickup on what doesn’t look right. They’d also miss out on learning how to take a problem and break it down so it can be done with code. That’s a foundational skill that takes time and effort to build, which is being farmed out to the LLM.

With super difficult algorithmic code, for me, it works best to just start writing and then fix, than to get stuck not knowing how to write.

Algorithmic code aside, I’m learning cuda now, and I just told gpt to write me some code and I debugged it then (with gpt’s assistance as well). It took me an hour to produce something that I expected it would take 2-3 days otherwise.

As for picking up what doesn’t look right - if it doesn’t work, you can pinpoint place where sth is messed up using a debugger/prints, and then you learn how the code works on the way.

I remember that when I was learning to program 30 years ago was the same - I rewrote pong game from a magazine, then kept changing and messing up things until I learned how it should work.

It’s like having infinite stack overflow examples (and explanations) to sample from and iterate on. It’s unlikely any one of them will solve a complex problem you’re working on but by reading and experimenting with enough of them eventually you grok it.

I think it would be quite mentally difficult to be successful at writing device drivers (any hardware device, many different fields) if a person had the slightest reservation about debugging. I've got past a lot of blockers by creative trial and error and making it work was the satisfaction driver. I actually enjoy the making things work part. I have had employees who felt the same, one of my jobs was telling management to fuck off while the person worked on the bug, for days without result if need be[1]. The "just writing code" part is how one actually gets to the interesting part where I earn my salary.

I have no idea how an LLM could help with that process.

[1] I successfully found a bug in a CFD solver (not a device driver) in the early days of templated g++ only after 2 weeks of fairly grueling half time work. Missing ampersand in an argument list. Believe it or not I was very happy and doubled down on the code. However! I once fucked up a job situation where my MPI-IO driver failed because I had a very subtle linux RAID hardware bug that I could not find after weeks of effort. I found it later, too late. That truly sucked. I really don't know how LLMs could possibly help with any of this.

LLM can't find a missing ampersand? Sad!

In 1993 or so g++ couldn't do it, but I suspect that all c++ compilers today would. Then why would I need anything else? However the point is that when dealing with proprietary hardware devices you occasionally get a situation where the incantations as "documented" should work, but they don't, and the usual software process diagnostics are silent about it. Some domain specific experienced creativity is required to coax a response to begin finding the illness. Yes you can pay for support and escalate but small shop management sometimes is a bit hesitant to pay for that.

I am very curious how an LLM is supposed to be trained on situations whose context does not exist on the open internet.

Perhaps only after an assignment where prompts and answers are shown, and most of the answers look plausible but have a bunch of subtle mistakes, and students need to determine which ones are wrong and why.

I speak English, not any code languages.

I recently learned javascript this way - knowing English and other languages (Thai, Mandarin), I kept pasting my pseudo-code, asking GPT to convert it into javascript, and then iterating on that. Took me a week or two to get the language without reaching for any sort of manual/tutorial. Speaking without experience - it would take me 2-4x as much time to learn the same thing otherwise.

:) My story is not entirely true, but close. My point being llms are learning language and logic (mostly English currently). Programming languages are just languages with logic (hopefully).

And if you think the ability to shorten meaning of a complex idea is exclusively the purview of code, think of a word like "tacky" or "verisimilitude"- complex ideas expressed in a shorter format, often with intended context with significant impact on the operations in the sentence around them.

How circumlocutious.

Maybe I’m dumb but I’ve never found a single textbook, manual or tutorial that taught me everything I need to know about a subject. I always have to make a circuit of multiple sources and examples before I understand fully.

> I recently learned Swift & Metal this way

I guess what you really learned is that the Apple-specific ecosystem technologies are so crappy, you always use a middleware to defeat them. Either Unity, React Native, or in this case, a 170b parameter super advanced LLM.

There's just no getting through to those Apple guys to stop making their own thing and being a huge PITA.

why is Swift crappy? I thought its a nice language

I know multiple very successful iOS developers and they uniformly hate react native and adore swift..

Both Swift and Mateal are beautifully clean - and I’m saying this as someone who spent years hating on curly brackets and static typing.

Ditto Xcode.

The biggest issue is the lack of documentation for Metal.

I don't do Swift or JS/React professionally (well, tiny bit, but I'm not a "React professional"). However, I've made non-toy Swift/SwiftUI* (~7k loc) and React (~4k loc) project.

Imho Swift/SwiftUI is way more of a joy to work with. In particular, SwiftUI is much easier to visually parse whereas JSX is noisy and IMO hideous. I also ended up needing to bring in way more third-party dependencies for JS/React, something I hate doing if avoidable**.

* It is my understanding that many iOS professionals prefer UIKit bc SwiftUI is still missing stuff. I only ended up needing to fall back to it once or twice.

** ClojureScript/Reagent do solve most of my issues with JS/React (by essentially replacing them). Clojure is well-designed with a great standard library (so less need for dependencies or implementing stuff that should be in a standard library). Hiccup is way preferable to JSX.

The assistants might be good enough to produce homework level or test level answers to contrived questions. But then utterly fall over in a real world codebase.

Reply on news.ycombinator.com