I Taught My 8yo Subduction Zones with NotebookLM | Hacker News Reader

I Taught My 8yo Subduction Zones with NotebookLM

timkellogg.me

122 pointstkellogg56 comments

I'm a CS/AI teacher in an engineering school. A few days ago, towards the end of my course on convolutional neural networks, I asked my students to explain why tha first linear layer of the example PyTorch network had a specific number of neurons. This is a non-trivial question whose answer isn't directly available online (it depends on the input dimensions and the nature of all previous layers).

They struggled for a while, and the first student who gave the right answer explained how he did it. All morning, he interacted with ChatGPT while following my course, asking questions each time my own explanations weren't sufficient for him to understand. He managed to give the LLM enough context and information for it to spit not only the right answer, but also the whole underlying process to obtain it. In French, qui plus est ;)

This was for me an eye-opening, but also a bit unsettling experience. I don't use ChatGPT & co much for now, so this might seems pretty mundane to some of you. Anyway, I realized that during any lecture or lab, teachers will soon face (or are already facing) augmented students able to check and consolidate their understanding in real time. This is great news for education as a whole, but it certainly interrogates our current teaching model.

"Check and consolidate their understanding" by reading generated text that is not checked and has the same confident tone whether it's completely made-up or actually correct? I don't get it.

>interrogates our current teaching model

Jesus, many many things put our current teaching model in question, chatgpt is NOT one of them. Tbh this excitement is an example of focusing on the "cool new tech" instead of the "unsexy" things that actually matter.

> by reading generated text that is not checked and has the same confident tone whether it's completely made-up or actually correct? I don't get it.

This is a valid point, but it's referring to the state of things as of ~1.5 years ago. The field has evolved a lot, and now you can readily augment LLMs answers with context in the form of validated, sourced and "approved" knowledge.

Is it possible that you are having a visceral reaction to the "cool new tech" without yourself having been exposed to the latest state of that tech? To me your answer seems like a knee-jerk reaction to the "AI hype" but if you look at how things evolved over the past year, there's a clear indication that these issues will get ironed out, and the next iterations will be better in every way. I wonder, at that point, where the goalposts will be moved...

No, ChatGPT and others still happily make stuff up and miss important details and caveats. The goalpost hasn't moved. The fact that there are specialized LLMs that can fact check (supposedly) doesn't help the most popular ones which can't.

Have you tried Claude.ai. In my experience on computer science topics, the LLMs are very good. Because they have been trained on a vast amount of information online. I just had a nice conversation about mutexes and semaphores with claude and was able to finally grasp what they were.

I do not know if this is the case for example for mathematics or sciences.

Is it possible you have not used ChatGPT recently?

>To me your answer seems like a knee-jerk reaction to the "AI hype" but if you look at how things evolved over the past year

It's not a kneejerk traction, like you said it's been 2 years of nonstop AI hype. I have used every chatbot model from openAI (3.5, 4, 4o, even o1) and a few from other companies as well. I've used code copilot tools. I've yet never not been disappointed.

> there's a clear indication that these issues will get ironed out, and the next iterations will be better in every way

On the contrary, there's NO indication of meaningful progress since the release of GPT 3.5. There's incremental progress, sure, as models get larger and larger and things get tweaked and perfected, but NO breakthrough and NO indication of an imminent one. Everything points to the fact that the current SotA, more or less, is at good as it gets with the transformer model.

> now you can readily augment LLMs answers with context in the form of validated, sourced and "approved" knowledge.

Not sure what you mean by this

I'm no LLM fanboy and I do know about their issues and shortcomings.

I also think that asking the right questions to a model while following a lecture, assessing its answers and integrating them into one's own reasoning is difficult. There is certainly a minimum age/experience level under which this process will generally fail, possibly hindering the learning outcome.

Nevertheless, I saw with my own eyes a mid-level student significantly improving his understanding of a difficult topic because he had access to a LLM in real time. I believe this is a breakthrough. Time will tell.

I don't know, seeming 'conversationally smarter' when having access to a language model is much different from just looking stuff up and pattern matching answers?

I'm afraid these models are making people sound smarter and feel smarter without any actual gains in real world problem solving skills.

The fact here is that a student, using ChatGPT, managed to give the right answer. And I agree with GP that the teaching model must evolve. The cat is out of the bag now and clearly students, of (unfortunately) almost all ages, are using it. It being "cool new tech" or anything else doesn't matter and as a teacher it must not be dismissed or ignored.

Not all subjects taught have to evolve in the same way. For example, it is very different to use ChatGPT to have a technical discussion than to simply ask it to generate a text for you. Meaning this tech is not having the same impact in a literature class and here in a CS one. It can be misused in both though.

I always come back to the calculator analogy with LLMs and their current usage. Here in the context of education, before calculators were affordable simply giving the right answer could have meant that you knew how to calculate the answer (not entirely true but the signal was stronger). After calculators math teachers were clearing saying "I want to see how you came up with the answer or you won't get any points". They didn't solve the problem entirely, but they had to evolve to that "cool new tech" that was clearly not helping there students learn as it could only give them answers.

But is it so much different from googling answers 20 years ago?

Granted, it can do the write up for you, but if google landed you in a flat-earther web in 2024, was just as useful as GPT today…

My gripe is that LLMs don’t reason, they search and parse, but is presented as reasoning.

The student isn't an idiot, they'd use what the teacher says as their ground truth and chatgpt would be used to supplement their understanding. If it's wrong, they didn't understand it anyway, and reasoning/logic would allow them to sus out any incorrect information along the way. The teaching model can account for this providing them the checks to ensure their explanation/understanding is correct. (This is what tests are for, to check your understanding).

How is someone who is learning something supposed to figure out if what chatgpt is saying is bullshit or not? I don't understand this.

It's a kind of Gell-Mann effect. When I ask it a question of which I know the answer (or at least enough to understand if the answer is wrong) it fails miserably. Then I turn to ask if something which I don't know anything about and... I'm supposed to take it at its word?

You have what the teacher has told you as your primary correct reference point (your ground truth). It should align with that, if not the LLM is wrong.

Obviously the gaps between is where the issue would be but as I say the student can think this through (most lessons are built on previous foundations so they should have an understanding of the fundamentals and won't be flying in the dark).

> "Check and consolidate their understanding"

Now do this with words spoken on a date or messages etc. Terrifying

I don’t know if you have been teaching, but I have (for nearly 19 years now ) to a lot of different people of various ages. I’m also a daily user of LLMs.

I’m firmly convince that LLMs will have an impact on teaching because they are already used in addition / superimposed on current classes.

The physical class, the group, has not been dislodged even after hundred of thousands of remote classes during confinement. Students were eager to come back, for many reasons.

LLMs have the potential to enhance and augment the live physical class. With a design school I teach at, we even have proposed a test program for a grant, where my History of Tech Design will be the in-vivo test ground of new pedagogical strategies using LLMs.

Tools that graft into the current way of teaching have had more impact that tools that promise to “replace university/schools”

I might be misunderstanding you, but the question you posed is all over the internet. First try, first page. It does not surprise me an LLM can “help” here.

My deeper issue with this tech is not its “knowledge”, it’s the illusion of understanding that I am afraid it fosters.

Lots of people will nod and agree when a competent teacher/mentor figure shows them something and walks them through it. They even think they understand. However, when given an actual new problem that they have to solve themselves without help they completely break down.

I am all for shallow learning as a hobby. Myself I love it, but I think it is dangerous if we misunderstand the nature of the problem here. Understanding is only partly based on consumption. A significant part of any craft is in the doing.

Take something like calculus. There are mountains of beautifully crafted, extraordinary videos on just about every nuance calculus has to offer and you can watch it all. It will give you a lot of concepts and this alone might be worth something but your time is better spent watching one or two videos and then practicing problems for hours.

My personal impulse was to grab to videos or books the moment I was stuck in my younger years. I now recognize how flawed this strategy was. Sure, it was “productive”. I got stuff “done”, but my knowledge was superficial and shallow. I had to make up for it later. By doing, you guessed it, a shit ton of exercises.

One thing I do appreciate is the availability of good quality content nowadays. Something like 3blue1brown is amazing and my university actually recommends watching his videos to supplement and ground your understanding.

No matter how many videos (or LLM podcasts) you consumed though, there is no way around “doing the work”.. as some painful questioning by any professional will quickly show you.

OP here: I definitely agree that shallow learning is an issue, and that it's an intoxicating effect. I've done it a few times — spent a few minutes learning a new topic, only to realize when I put it into practice that I'd been lied to.

But that's why it's critical to engage kids in this. There's a skill in using AI. Resisting the urge to take it at it's word, yet still using it for what it's good at. You can't build a skill without practice.

The fun side of ChatGPT is that if you probe it for information like this, it'll also generate complete fantasy. Without an expert to consult, the generated explanation may as well conclude the earth is flat and the sky is green.

No, this is not accurate in my trials. I use Claude.ai daily. If you ask questions on niche topics or dive down too deep, it says that resources on the topic are limited and you should consult a book.

I'm curious to hear more about this. I've seen very little hallucination with mainstream LLMs where the conversation revolves around concepts that were well-represented in the training data. Most educational topics thus have been quite solid. Even asking for novel analogies between distant and unrelated topics seem to work well.

I haven't messed with it in a few months but something that used to consistently cause problems was asking specific questions about hypotheticals where there may be a non-matching real example in the dataset.

Kind of hard to explain but for example giving a number of at-bats and hits for a given year for a baseball player and asking it to calculate their batting average from that. If you used a real player's name it would pull some or all of their actual stats from that year, rather than using the hypothetical numbers you provided.

I think this specific case has been fixed, and with stats-based stuff like this it's easy to identify and check for. But I think this general type of error is still around.

Thanks, that makes sense. I avoid using LLMs for math because it is only a text token prediction system (but a magical one at that), and can't do true numeric computation. But making it write code to compute works well.

The important bit here is that you spent time with your kid and engaged with her, not the tech that you put in front of her.

Neal Stephenson, when asked on how to make the Young Lady’s Illustrated Primer real: “Kids need to get answers from humans who love them”

A point the book explicitly made…

I don't know why people immediately try and shut this kind of thing down.

Sure, them spending time together is important. However for what happened here so is the podcast about subduction zones at a level the kid could take something from.

We don't actually like tech or hacking here. I'm not sure why it's called Hacker News anymore. Feels like this post is right up HN's alley, then I see a comment like theirs.

There are so many benefits to AI, that I've directly experienced in how it's transformed how I can interact with videos, books, and research/learning in general, and it feels like half the site is just "nuh-uh". It's like low-key gaslighting.

> We don't actually like tech or hacking here.

I think that's a very ungenerous reading of the parent comment.

Sure, LLMs are useful (I use them to digest/filter larger bodies of text myself, several times per day), I'm not denying that. But, if we had to remove one of the two aspects discussed in the article (and the comment), which one would be more valuable to the kid: the attention of their parent or having access to notebooklm? As someone with a hacker attitude (fwiw) I find less technical or dumb solutions to problems more interesting.

Like, no one is saying that the LLM wasn't useful here. Both can be true at the same time. What's interesting is how these two concepts relate.

I think their comment is very ungenerous of what this kind of thing provides in being able to engage in a dialogue about a particular thing. It can be insanely difficult to break things down for somebody unfamiliar to a particular topic especially if you yourself don't have copious amounts of knowledge and practice in it. I repeatedly see people fail to understand the benefit of something like this because they refuse to want to.

> which one would be more valuable to the kid: the attention of their parent or having access to notebooklm

Sorry, this is stupid. As a parent, you want to look for tools and methods to help teach the child new things. Something like this is insanely helpful and it isn't a question of "would you rather have a parent or would you rather have notebooklm". Nobody is suggesting we replace parents and nobody is going to give a child these choices. It's a complete misrepresentation of the situation and distracts from the benefits, as a teacher or as a parent, of having access to tools like this.

My god, it's frustrating having these idiotic conversations every time LLM comes up.

> But, if we had to remove one of the two aspects discussed in the article

This was never up for debate until that comment. It's stupid and unproductive.

So many times I see parents and teachers look for robot kits or books or courses or other types of kits (Arduino anybody?) to help children engage with new topics and learn new things. And people are always very supportive and helpful in these kinds of things, because it's a GOOD THING to want to help children.

But I never see somebody be like "but what's actually important here? The robot kit or the parent being there?" NOBODY ASKED UNTIL THEY DID. It wasn't up for fucking debate.

Why is it when it's LLM that suddenly this is so important to point out? Why don't I see that when somebody talks about how great it was to go through an Arduino kit with their kid?

I'm not sure where you saw the intent to shut it down - OP is doing a great job being involved with their kid, if only every parent was like this! - just making sure people don't get confused about what is going here, as we routinely see (including on this very site) tech people who genuinely believe that human teachers can and should be replaced with AI.

The NotebookLM podcast feature is so unbelievably good! I've been feeding it random papers I've always wanted to read, and it's giving me these entertaining high-level summaries. The banter is almost addictive. Also, reading the papers in detail is quite easy after you understand the topics.

Language model summaries can change meaning in hard to detect ways unless you're a very knowledgeable specialist.

So does your brain.

I fed it a pretty short article (~8000 words), and multiple times it said literally the exact opposite of what the article stated or made parts up entirely. I'd suggest reading and understanding whatever you give it because it will almost certainly lead you astray.

And even when it is technically right it often goes off on tangents not in the paper -- for instance I fed it one of my papers on a pipeline to analyze microbiome data and it spent half of the "podcast" on talking about how important the microbiome is to health and how health care needs to take the microbiome in mind. All true, but it doesn't have to do with the pipeline! That being said, I'm blown away by how realistic the voices are and how they often interrupt and talk over each other the way that real human podcasters do.

I am really impressed by the podcast it generates and I hope they will introduce other voices/styles because this style can be too much for some people (including me). Not so impressed about the chat experience though. Sometimes it sounds the bot is annoyed at my questions :) Which is a refreshing change from the pathological politeness of ChatGPT.

I'm looking forward to this expanding to other languages. There is so much great information out there, but unfortunately mostly in English. LLMs are really great at translating, and together with recent text-to-speech, seems straight-forward for the tool to consume inputs in English and do the podcast generation in another language.

Would be a great step in spreading knowledge across cultures and backgrounds.

It’s crazy how good NotebookLM is at summarizing information. I've been using it non-stop to quickly get through a pile of articles I usually put off, and it’s honestly freeing up so much time.

It's not just the saving time part but it feels like I'm actually getting more out of what I read. Makes me wonder what we'll be able to do with this kind of technology in the next few years!

Tectonic, not "techtonic."

And poor kid, being fed this AI sludge. If you want to teach an 8-year-old about subduction, use the Magic School Bus!

lolololol, no that's me. I don't use AI at all while writing, which is why I make these mistakes

I also find that words I type commonly (e.g. "tech"), get so wired into muscle memory, that they sometimes show up where they don't belong!

I've been trying to add high quality elevation data to the maps on my blog. I'm way out of my depth trying to convert things from one format to another. There are seemingly endless ways to store the data and a bunch of acronyms (DEM, DTM, DSM, SRTM etc.)

Anyway, I resorted to ChatGPT to help. It gave me a complete end to end process, including how to install tools, that was almost entirely wrong at every step. But, in one step it pointed me towards a tool I didn't know about, and that too finally plugged the gap I needed.

I was able to do this because I treat ChatGPT like any other source of knowledge: with a healthy dose of scepticism. Almost everything I do is piecing together knowledge from different sources, written at different levels for different audiences etc. What worries me is whether kids are going to miss out on this. Seeing a fully typeset document or listening to pristine audio of someone who intentionally sounds so confidently correct seems like it will really fuck with their ability to discern the truth. They'll either believe everything, or they'll believe nothing.

What worries me even more is these models are controlled by an oligopoly who twist and bend them to fit their own beliefs and narratives. Instead of going to the public library children will be getting knowledge filtered through layers of "AI" that will never tell them anything it's not supposed to tell them.

Thanks for sharing. Does NotebookLM covers other languages as well, for instance Urdu, arabic or Farsi?

Right now the podcasts are only in English, and (I believe) only use English sources. I'm sure this will change soon enough though.

I fed it an Italian Wikipedia entry and it generated a (relevant) podcast and summaries in English, so at least the input seem to support multiple languages even if the output is in English.

If I had come across something like this 2/3 years ago it would have looked like magic, but even now it’s quite impressive.

Edit: You can change the language following the indications in the FAQ; after that, text generation seems to be in the configured language, but the podcast is still in English.

I gave it a book in Japanese that I wrote and it generated an accurate podcast about it in English.

Thank you Stranger!

I have been seeing lots of articles about NotebookLM, but had never tried it. Today I am going to try this (with a different article) for my 6 year old daughter. This looks super amazing to teach tough concepts to kids

Thank you!

Indeed! The only downside is that my kids don't speak English yet...

On a parallel note - came across this damn useful use case of NotebookLM : atomicideas.ai - creating engaging audiobooks from the book summaries.

Is NotebookLM the most innovative AI product from Google in the GenAI space?

Note sure about what I think of the ethics of what Atomicideas 2.0 is doing: taking classic literature and making that a Q&A chatbot, for a fee.

Impressive! I'm wondering if something like this is possible with other models, because of Gemini's extremely large context length certainly helping a lot with parsing such large documents.

This is amazing. Would love an actual geophysicist to chime in to verify how accurate this was. I really felt like I learned something...but did I?

Dupe: https://news.ycombinator.com/item?id=41696561

Wow, I honestly was not intending to click through and pay much attention to TFA, but I pressed "play" just to see, and I was honestly immediately hooked.

Maybe because I'm saturated with LLM stories, and yet another summarization use case, blah blah blah.

There is something fascinating here to me that two (synthetic) people talking to each other engaged my attention, and then amazement that they were turning this esoteric paper into a radio show at the popular level. (or for any paper whose content would never actually warrant two real people making a radio show about it, if you could even pay them to do so) Maybe an aspect of it is that if two people are talking about something, it inherently has passed a filter that "it might be interesting" to know about.

Surprisingly eye opening about the technology here.

[originally posted to a previous submission: https://news.ycombinator.com/item?id=41696561]

Honestly, the resulting "podcast" is kind of underwhelming as an actual artifact. It's basically "plate tectonics for kids" mixed with some gobbledygook from the input paper. Sure it's kind of neat as a tech demo, but to actually try to use that just show the mediocritizing to enshittfying influence of LLMs.

Honestly, I think the guy would have gotten an even better response from his kid from any competently done "plate tectonics for kids" video (e.g. from Khan Academy: https://www.khanacademy.org/science/middle-school-earth-and-...).

Curation and digital distribution basically solved this problem earlier with better results. LLMs are just a shittier private label article factory version (https://blog.smashwords.com/2010/01/scam-of-private-label-ar...).

This article could be improved by deleting the emojis from it.

Reply on news.ycombinator.com