GPT makes learning fun again
vipshek.com
vipshek.com
I doubt either of those wishes are going to come true though. Search engines are likely always going to be SEO'ed into uselessness and GPT isn't intentionally telling lies.
In an ideal world, people would learn only from high quality sources like books and schools. But a shocking amount of people learn mostly through social media and whatever they find on Google.
I think LLMs will provide a better alternative.
ChatGPT’s current super power is helping people get from 0-to-1 on a new topic. In particular if that topic is adjacent to or a different niche with your expertise.
It’s not currently amazing at taking someone from intermediate to advanced knowledge.
At least in my experience. If I’m using a new library/framework/API for the first time it’s amazing at answering the endless newbie questions I have.
My suggestion is to stick with it and get a feel for what it's good at.
I've found that after a few months of using ChatGPT every day I've developed a pretty solid intuition for which questions are likely to get good answers and which are likely to trigger hallucinations.
It's difficult to describe what those intuitions are though!
One rule of thumb I've developed: if something is likely to be "common knowledge" - if it's something that is likely to have been discussed accurately on the internet by many different people - then ChatGPT is very likely to answer questions about it accurately.
Uh, and sooo boringly, especially if there's even any tiny part of it that is developing or theoretical, and you want to learn about that part.
I've used the phrase/request "more obscure (perspectives/explanations/etc)" with GPT so many times.
(Try it, you might be surprised at alternative takes on things, takes which are not even necessarily conspiracy theories and such)
...Which has made me think: Maybe life is more boring, the more one thinks there are just really good one-and-done answers to most everything.
One of my favourite is to ask it to explain with analogies. The other day I was digging into the attention mechanism used to train LLMs, so I asked it:
"Explain queries, keys, and values in the context of LLM attention using analogies from Terry Pratchett's Discworld"
I find sometimes I get some real gems out of this that help me remember things much more effectively than just reading the basic explanation.
(In case anyone's curious I ran that just now and got the following: https://gist.github.com/simonw/777b1d19f36beb39fb4216a0238fe... )
Most everything has been covered in logical extensions of concepts started in the 80s-90s. Those products aren't good for our..
Ah there's no point commenting here anymore. HN is so blinkered in it's thinking.
The feeling of flying high on lofty concepts and pretending to get a bird's eye view of tech is no longer worth the squeeze. The thought patterns here are predictable like slashdot. I don't belong here. Bye.
You've probably heard this before, but IMO you'll never find a happy place working from that perspective.
Recommend 10 project gutenberg books that are fun to read --> same old stuff--Huck Finn, Robinson Crusoe, War of the Worlds.
But I'm not sure if this particular use is really a temperature-relevant thing or not.
If so, this information is already easy to find, making GPT redundant.
If so it seems like 1. that problem is of our own making and could be fixed without ChatGPT and 2. what will actually happen is that 'experts-exchange' and all the similar slightly scammy help sites and forums are going to try to stop LLMs from stealing their lunch.
You can ask it to "combine" knowledge in "novel" ways that are not discussed verbatim on the web. It's not groundbreaking reasoning by any means, but it can be very useful. ("My ridiculously specific question about model X83844-QQ combined with random factor X")
I asked chat gpt to review the topic for me and it wrote a helpful summary that tracked with what I remembered and provided enough keywords that provided better search results - fact checking and digging for more detail became much easier when I was able to find the exact wikipedia page I needed (and other resources).
Personally, trying to use it to write code (primarily Elixir backend and Rust systems/CLI), I tend to run into:
- Hallucinating APIs that don't exist
- Hallucinating entire libraries, despite being told repeatedly they don't exist
- Saying it will make requested changes and not doing so
- Not being anywhere close to idiomatic code
- Not being able to explain code it writes
- Running out of "memory" (I can't remember the right term. Context?) in the middle of generating code, then telling me I never prompted it when I ask it to continue
On the other hand, I've found that it's good at cleaning up ugly data. I can copy/paste in a table with bad formatting, ask it to turn it into code, and it does it near-perfectly. That's been the best use-case for it I've found so far.
I use boring normal free ChatGPT so maybe it's on me for not using GPT-4 or some other model, but either way, imo it's not been very impressive in the problem spaces I find myself in.
I've actually found it (ChatGPT-4) reasonably good at explaining other people's code, for what it's worth. It does require providing some context for the code it's explaining. ("This function is part of library X that does Y, and it seems to be about Z. I can't figure out the purpose of the loop in the middle, though. Can you explain it?")
I'm with you on it writing code, though. It makes things up too frequently to be really useful.
ChatGPT is also much better for Python and JavaScript than Rust and Elixir, presumably because it's seen an order of magnitude more example code for those languages.
You can use GPT-4 fairly cheap if you sign up for a developer account on platform.openai.com and then use their playground. There you pay per usage; even though I've been using it fairly heavily, my typical monthly usage is still way under $20.
Here's an example that both impressed me and saved me a load of time just yesterday:
Some of the “issues” are more general, as in it giving you dated answers. I asked it so build some ODATA things in Typescript and C#, and it did to varying degrees of success, but some of the code was deprecated. Some ranging to “you should never, ever, do things this way” to “well IActionResult was replaced by ActionResult but it hardly matters”.
Others was where it made things up once pressed upon being wrong. I think the two most hilarious situations was when it did its “Sorry, you’re right…” thing and then proceed to give the exact same answer it had just given prior. The other was when it made up a function that had never existed, it has a very convincing name and at first I thought it was again a matter of something deprecated, but it turned out to be completely made up code. Some of this is down to me not prompting it right, but part of it is also worrying. Because what really made me worry about it was when I joked with it. I read (listen to) a lot of Warhammer audiobooks from Black Library, and, when I jokingly asked it something silly about Khorne at one point, it led me down a rabbit hole of discussing books with it. Books it obviously had never read, but would still confidently tell you about wrongly. Maybe it got its knowledge from the internet, maybe it made it up, but what was interesting to me about it was that if it would be so confidently incorrect about those books, then what else would it be confidently incorrect about.
This isn’t just a GPT problem of course. If you want to learn something, you need to consider the sources you use. When I went to folkeskolen (school for children aged 6-14) we were taught the pyramids were build by slaves. Later that has been disputed because of how well fed the workers were, but if you wanted to learn about the pyramids and you read my old school books then you wouldn’t get the updated knowledge. Similarly a lot of the sources you can find for learning programming are outright terrible, but outside of GPT you tend to be presented with a myriad of choice to remind you that some sources are better than others. With GPT the source or what it teaches you isn’t obvious. If you wanted to learn about the pyramids, you probably wouldn’t pick a 30 year old school book for children after all.
Stop using GPT as a database! GPT is far more useful a reasoning engine that can accumulate fuzzy data and then provide various views or transformation of that data.
So asking GPT to parse a Wikipedia page and then asking it to teach you from it - this is a much more successful usage than what the author in the original article is doing.
It is not useful as an accurate source of information. It’s inaccurate sometimes, and it’s hard to tell when. OTOH, as a formatted, it has some actual world-changing potential.
Reasoning is not its strong point, IMO. It's a next-word prediction model, why would it be? It's doing what an LLM does. Frequently, nonsensically.
My team has been working on several advanced techniques using reasoning on LLMs. The stacked performance of all of these techniques combined yields is quite impressive.
I just tried using GPT-4 to translate a random htaccess file I found to nginx and it seemed to do a good initial job: https://gist.github.com/simonw/8174f653a0c08c120830c56332054...
Every second that I didn't have to:
- go look up a similar problem on Google
- open three or four stack overflow tabs
- read through the stack overflow links
- copy out the answer
- change all the variables to match variable names that I want
All of this represents a huge time saver for me. I'm honestly baffled that people lack the ability to use LLMs in a productive and optimal manner.
in surprised how few services developers I’ve encountered who don’t know hot to configure a web sever or LB.
ChatGPT will only help you with the simplest of tasks, and even then you're gonna fail if you don't know to correct it. If you go beyond the most basic of the tasks you can learn how to do in a few hours, good luck fitting them in a chat prompt.
The amount of Kubernetes I needed was probably more than what ChatGPT can solve, but at the same time, I spent an awful amount of time looking at various templating solutions.
Docker is being replaced with Podman. Supposedly a drop-in replacement, but the tools built for Docker don't play nicely. You have Ansible, but there's also Puppet, Salt, Chef. There's Terraform and also CDKTF (from the same company too), Pulumi, plus tools from each cloud provider. There's Prometheus and Nagios, Datadog, Sentry, New Relic.
The irony of CI/CD tool of your choice is not lost on me.
I'll give you Kubernetes itself as the system.
Yes, the concepts (containers, orchestration, reproducible deployment, infrastructure as code, unified monitoring, continuous integration) are well-established. How you get there? Not at all.
Yes, the OP was hyperbolic when claimed the tools change every year. But the landscape of devop tools is changing. There are dozens of dev tool companies funded in the 2023 winter batch of Y Combinator alone. Someone sees potential to disrupt the established tools. And it will be disrupted.
There was a world before Kubernetes and Terraform. It wasn't that long ago.
https://www.ycombinator.com/companies?batch=W23&tags=Develop...
> Saying that DevOps tooling changes every year is absolute nonsense.
And then you list 5 technologies with an alleged lifespan of half a decade, which sort of hints at the idea that someone would need to learn/update a new tooling skillset once a year.
In that light, GP's comment doesn't sound so wild.
Huge +1.
I’ve got it to do a large part of my work for me by chaining some simple API calls.
The fundamental conceptual gap I see - people often ask it to do some “thinking”. Then are annoyed by the inaccurate output.
A simple example is doing word count and it getting the wrong answer very confidently.
Of course it sucks at that. It doesn’t have a counter internally. But if you ask it to number each word in the input and output a list, then ask for the word count, it gets it right every time.
Almost like how a human might count words manually.
Here is what I asked it to do the other day, and it just did it right away and got it entirely right.
> Write async tokio Rust code that takes in a Vec<u8>, writes it to a temporary file, calls /usr/bin/svc infer pathToTemporaryFile, reads and deletes temporaryFile+".out" and returns Result<Vec<u8>> containing the contents of temporaryFile+".out"
I knew what I wanted, I could have written it myself, there was no unknown but it took fewer keystrokes and honestly when asked to write small components like this with pedantic detail, it does an incredible job and the output is easy to validate quickly.
As someone who is far from an expert in DevOps, this has saved me a lot of time recently, and it worked fairly well to get past duct taping infrastructure together to then leverage.
In my experience, GPT saves me some time when I don't really have to think about idiosyncrasies anyway. In cases where I do have to think about them, GPT doesn't offload that from me. It often tries to mash unrelated frameworks together or makes up non-existing APIs that it would find useful for the given task. When it does work well, it mostly boils down to automated series of copy'n'pastes. Saves a bit of time, sure, but doesn't really transform the way I work.
- Article: https://simonwillison.net/2023/Apr/15/sqlite-history/
- ChatGPT transcript: https://gist.github.com/simonw/1aa4050f3f7d92b048ae414a40cdd...
If you just need the beginner's gist or are completely unfamiliar with something, they're not bad. Getting specific answers to corner cases is often a waste of time.
You always have to walk back and do more research to find the best practice.
And it doesn't learn. I stopped counting the times I told it it was wrong only for it to acknowledge the error and provide the same answer as a fix.
ChatGPT is nice to learn about new ways of doing things, new ways of composing things, but then a human must intervene to make sense of the mess that was generated.
If you want to make accurate predictions, you'll need reasoning or at least some process that approximates it.
"There a no bananas. Max is looking for bananas. Will he be successful: [yes/no]"
Tbh, it isn't surprising that it fails, its surprising that it has subjects where it is very useful.
So are you. https://ceoln.wordpress.com/2023/03/31/its-just-predicting-t...
Counterpoint: https://the-decoder.com/why-large-ai-language-models-dont-le...
Qualifications are a great way of figuring out who's worth listening to, but that's not always the case as AI/ML folks prove with regularity, but it's nice to know if someone isn't just an armchair commentator and isn't peddling baseless information.
That's not what it's claiming at all. It's taking umbrage with the word "just" in "just predicting the next word".
A gun "just moves some metal from one place to another".
I'd sooner listen to people who are leaders in their field when it comes to LLMs than someone rambling on their free WordPress blog.
It is just an LLM. That's an objective fact. IBM's Watson is a pipeline of different algorithms. Neither are like a human brain, but multiple processes are going to be more brain-like in concept than just an LLM.
Followed by the link. You made the claim that I'm an LLM.
What actually is your point?
Hilarious to read this advice when GPT, by it's own designers, cannot reason.
Look up the Sam Altman podcast with Lex. He specifically talks about reasoning engines.
I like to think of consciousness as not the substrate, but the information flowing through it. Not the H2O molecules in a river, but the current. We're like eddies and swirls in mountain streams, our thoughts are the patterns of flow, not the unmoving rocks that set them up.
In this sense, when an LLM like GPT produces output, it's like a loop that has been cut, a single pass through what would be a circular process in a human brain. It can take a fixed number of steps from a standing start, but no more. This is because there is information flowing through it, transformed step-by-step. It just doesn't recirculate, it always exits after a fixed number of steps.
Think of an unrolled for loop that always exits. It takes a few steps, which look like a loop, but can never actually iterate.
I find GPT4 useful, but I still feel its wishful thinking to call it a reasoning engine. It is, in the end, a giant model for predicting the next word in a corpus which you condition with inputs. I find it most useful to think of it as exactly that.
Its unclear to me how long we'll have before LLM Engine Optimization is a thing and OpenAI/MSFT "need" to turn on their LLM profitability spigot; and what ChatGPT will look like then.
That said, I'm curious as to whether technically LLMs are inherently more challenging to game than search engines.
see: windows 11
Edit: Perhaps Colgate-Palmolive could advertise Palmolive on ChatGPT by having it so that, anytime liquid dish detergent does come up naturally in conversation, then the chatbot subtly points to Palmolive brand. But success here would be hard to measure.
- Coke Zero
- Tide Pods with Oxiclean Turbo
- GEICO Renter's Insurance
...
This ^. Prompting google is much more intuitive than prompting a chat bot. Also results are instantly available and you get more options to chose from. You can also filter out information much easier instead of having it summarised by a closed box that decides what's best for you.
If the information on ChatGPT is averaged out the chance of it being correct is high.
If you ask questions with bias in them you can sway the results of ChatGPT. That just means to me you’re asking the wrong questions.
In terms of devops. Programming. Configuration. Daily tasks. Generalised workflows. How-tos. Etc. ChatGPT will more than likely give you accurate results that are useful, or pretty damn close.
Obviously with information being limited to 2021 then it can be hard to find some solutions. I needed to do something with named pipes in c# that I couldn’t get working in .net 6. The examples it kept giving me were for .net core 3.1, or .net framework. When I asked specifically about .net 5 (since it doesn’t know .net 6 is released or that 7 exists) it apologised and said the code won’t work in 5 and gave me a working example in .net 5. Which identified that there was an api change I wasn’t aware of.
No amount of googling found a solution for me.
From [Make Something Wonderful: Steve Jobs in His Own Words][1], Steve once said in a interview in 1983:
> The problem was, you can't ask Aristotle a question. And I think, as we look towards the next fifty to one hundred years, if we really can come up with these machines that can capture an underlying spirit, or an underlying set of principles, or an underlying way of looking at the world, then, when the next Aristotle comes around, maybe if he carries around one of these machines with him his whole life–his or her whole life–and types in all this stuff, then maybe someday, after this person's dead and gone, we can ask this machine, “Hey, what would Aristotle have said? What about this?” And maybe we won't get the right answer, but maybe we will. And that's really exciting to me. And that's one of the reasons I'm doing what I'm doing.
And this future, expected "next fifty to one hundred years", is somewhat here already.
I always finish up by asking GPT to test my knowledge with a single-choice questionnaire. What I've observed is that the retention of the material is higher compared to "traditional" techniques. Perhaps the conversation style is more immersive, or perhaps focusing on specific knowledge gaps makes for accelerated / personalised learning.
There is of course the problem of accuracy, but I feel like it's often over-stated. Even if GPT is not correct at times, it often uncovers concepts and relations that paint a better overall picture for me, and lead me to better questions and follow up actions.
There seems to be less next-level analysis: which topics are more prone to inaccuracy, does the critique loop actually help LLMs overcome those inaccuracies, and do the benefits of LLMs outweigh the consequences of these inaccuracies?
That’s a positive thing about this generative AI revolution that I haven’t really thought about in those terms until now.
People go on and on and on about "accuracy", completely ignoring that accuracy is irrelevant to 99.9% of things that humans do in their everyday lives.
Simulating (positive) human interaction is far more impactful than getting facts correct.
In my experience, people with the attitude that AI will give them a more fulfilling relationship than humans will aren't exactly friendly to begin with.
Agreed. Look at all the responses in this post attacking the author for how he learns, it's embarrassing to read. Now imagine that person is actually a teacher or TA, or worse a co-worker. I'd much rather deal with an imperfect ChatGPT session than that kind of flippancy.
On that note, search in that regard always reminded me of those times where you ask a teacher how to spell a word and they say to look it up in the dictionary.
To learn anything useful on the internet, you pretty much have to skim. So much of the internet is so loaded with filler and BS that it is hardly worth reading at all.
With ChatGPT, it’s incredibly refreshing to be able to ask a question and get nothing other than a concise answer. No skimming required. I feel so much more focused and better able to learn this way.
Otherwise ChatGPT is prone to injecting lots of filler text as well.
OP learns in a way that's very child-like. When you are a five-year-old it's okay to learn by asking everything. That stops being acceptable by the age of fifteen. OP hasn't learned any research skills yet, and when OP's needs inevitably exhausts the ability of LLMs, OP would be utterly unable to read an encyclopedia or a research paper or perhaps a textbook.
A lot of the paper is talking to peer and people wanting to verify the validity of the paper - by it being peer reviewed, you can mostly assume that the paper is valid, and stick to what the paper is saying instead of it's methodology.
What? Is that something you can still assume in 2023?
you should check this
https://en.wikipedia.org/wiki/Replication_crisis
https://en.wikipedia.org/wiki/Data_dredging or P-hacking
and many more...
While this is valid, this shouldn't make us paranoid of all papers. If it's a paper out of your expertise, it's unlikely that you would be able to catch problems with it that the peer review process wouldn't have caught.
So in practice, it doesn't really change the way you interact with papers - it should change the way people write them and how the peer review process works.
If OP reads your comment, they will be no better at learning than they were before. In that way, it's a pretty unhelpful comment.
If I want to learn about topic C which requires knowledge of topics A and B, but C can also be generalized to concepts X and Y, it will be very hard to learn from Wikipedia.
If I don't know how to add numbers and look up "sum" on Wikipedia, in the second sentence I learn that summing is used for functions, vectors, matrices, and other things I don't know about. This is a cool feature and I love it for exploring but hate it for learning things that require a few layers of concepts to get.
Textbooks do the opposite and are awesome. An electronics textbook will take you step by step through all the concepts to get to LEDs, without "forward references" to the concepts you haven't learned yet.
The "problem" with textbooks is that it will take a while to get to the destination. LEDs might be in chapter 15 and you may not want to spend a few months going through chapters 1-14. You don't know what you will need to understand chapter 15.
But you can perhaps work backward - you are guaranteed that any unfamiliar concept introduced in chapter 15 will be covered in chapters 1-14, and that there is no rabbit hole.
ChatGPT or a personal tutor can shortcut this by giving you just the "narrow path" of knowledge to understand the concept that you want to learn.
I completely agree that for the latter goal, the approaches in the blog post are insufficient, even undesirable. And I do worry that the way I engage with content on the web is weakening my ability to go deep on a subject I'm interested in.
But I do think there is value in just being able to indulge curiosity quickly and consistently. Not only is it rewarding in its own right, but it also provides the spark that leads you to eventually go deeper.
Lately, I've found myself sitting at a laptop with friends, asking GPT a question, reading and discussing the response, and then coming up with and asking followup questions as a group. I don't think we would've done that in the past, because the interface of search engines and webpages and browser tabs were too unwieldy to engage with collectively. It just feels like a completely new way to learn things, and what's what I'm most excited about.
> just out of curiosity, I wanted to learn more. I get that LEDs consume less energy and release less heat, and that they're made using semiconductors. But what kinds of semiconductors? How do semiconductors work in general, anyway?
And they proceed to type "LED" into Google. Why not "led what kind of semiconductor" and "how do semiconductors work in leds"?
I assume, OP didn't write "LED" in the ChatGPT text box without any context either.
I did try Googling “how do LEDs work” for comparison, but it yielded the same top few results. Of course, I could have iteratively tried different search queries to get to the answers I wanted, but this gets at my real point: I don’t have to formulate 5 different search queries anymore, allowing me to maintain one focused line of inquiry. I talk about this a little in the “fewer browser tabs” bit of the post.
I do think someone could create an alternative search UI that would be better for learning on the web. Something where you can run multiple searches and “collect” the useful information you find into a single page, rather than having the results split across a mess of browser tabs and note-taking windows. Maybe I just find juggling many browser tabs more annoying than other people do?
Anyway, I tried the queries you posted above, and most resources I found were still very confusing for a layman. The one exception is this page, which I think does a great job of introducing additional complexity on this topic gradually: https://electronics.howstuffworks.com/led.htm
Right now, if the level involves advanced math, it’s better to switch to other sources at some point, but that will change.
You can ask GPT-4 to tutor you, also.
I think this is partly why I'm still looking to be wowed by this technology, personally, in terms of what it can accomplish for me. And while it could be rightly said I've made things unnecessarily hard for myself approaching life like this, I feel it has been beneficial, and enriching, to force myself to really ask, what is this person saying here? In particular, I wouldn't want GPT to lead to a general lessening of empathy.
I personally don’t believe there’s mutual exclusivity here.
I spent the first half of my life just absorbing. I’ve spent the 2nd half of my life so far undoing the patterns of thought that can result from staying inside one’s head instead of engaging with people.
In my experience, asking the person saying something what they mean is far more effective than asking myself what they mean.
When I ask them, I can form real empathy.
When I only ask myself, I just waste significant energy on all of the ways I imagine I could or should be empathetic.
> In particular, I wouldn't want GPT to lead to a general lessening of empathy.
While I have a lot of concerns about LLMs and the future of literacy, propagation if misinformation, etc, I don’t think ChatGPT is any more risk to empathy than 100 other aspects of modern life.
Facebook, Twitter and Reddit seem far more responsible for the erosion of empathy, and it’s unclear how LLMs would inherently lead to a “general lessening”. I think that ship sailed a decade ago.
Put another way, if you’re going to sources other than the individual speaking to clarify what they’re saying, the underlying issue is probably not ChatGPT or whatever the next tool is that comes around.
Another form of what you describe is leaning on one’s friends/acquaintances. Plenty of people do this, often with poor results. Reddit’s various relationship forums are a great example. I translate what I thought I heard and ask a 3rd party who wasn’t there what they hear. But by doing so, I remove even more context and make it even less likely to arrive at a useful answer.
I’m sure people will use LLMs for this, but the root issue is deeper, not caused by these tools.
I think that with time, we’ll get better at determining which types of conversations are worthwhile and which aren’t.
If I’m trying to understand a complex multi-faceted technical issue, it’s amazing to be able to drill deeper and deeper into the knowledge contained within the LLM.
If I’m trying to understand the internal states of other people, I have no reason to believe I’ll find good answers in a model that wasn’t trained on that person’s thoughts.
If I want to try Rust, I don't want to be taught uint8 v uint16 or that you shadow variables. I want to know the interesting parts.
ChatGPT is pretty good at this and the other thing I want: pandas training. You can ask it to generate exercises at any difficulty and also provide test data!
This tool is the biggest mind expander for me since search engines.
I tend to get better answers from their "automated questions" which are paraphrased versions of my query. So it clearly understands what I'm after.
In order to promote diversity, i would recommend perplexity.ai which offers a similar experience as chatgpt (I'm not affiliated and i have no clue what their tech stack is like) It also offers links back to pages and follow up questions etc. Highly recommended if you need to learn something new and you don't want to bang your head on the keyboard googling or ddg'ing
I'll give an example. I recently needed to learn about k8s, minikube, kubectl et al for a project. I had some vague idea about the tech but nowhere near enough for what i needed to do. Google was useless because it kept taking me to doc pages which is like being lectured but i needed specific information. Perplexity was amazing in helping me with the right bit of information, example code AND links if i do want to read further
GPT is like that "know-it-all" friend we have who just has something to say about anything, with knowledge skimmed from the internet.
GPT is a language model. It outputs what you want to hear, not what is correct.
Nonsense. How could it possibly know "what you want to hear"?
"How do I do this"
"How do I do that"
"What is this"
"What is that"
Maybe a better rephrasing would be "ChatGPT has been trained to give answers to questions in a clear, confident manner, regardless of the content"
From the announcement post: https://openai.com/blog/chatgpt
> We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant.
So I agree with OP; it's been trained to give answers that sound plausible but not necessarily correct. It's even mentioned in the "Limitations" section at the bottom of the blog post.
> ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
I think we need to think about how to keep these valuable sites going, because they are ultimately providing most of the value of the various available language models.
PS you would also need to do this if you started with Wikipedia as well.
Search engines are much better On uncovering guides on that, specially from experts and verified sources. It's a bit of work to verify that, but then it is part of learning itself, not sure trying to punt on that to fast track your learning your whatever is going to make a meaningful difference in terms of time
setting up a venv.. environmental variable issues in windows. *diagnosing a UTF-8 issue in windows.
i get that professionals problems would be harder to answer...however, getting responses without wading through stack exchange entry after entry has really kept me focused, and prevented the often times frustrating recursive spiral which is getting an issue with your issues issue...
It didn't give a wrong answer when asked "Is there a digital to analog converter with an 8V analog range and serial input?", which another poster (mhb) had shown to trip up plain GPT4.
Although it's a little weird that we're on a board where someone could seriously mean what I said and people wouldn't think it was a joke. e.g. the "You let your kids use calculators, don't you?" crowd.
It's also kinda rude to assume someone is joking when they were actually serious, so yeah one tends to get taken seriously when in doubt...
Because if you don't, you won't understand the answers.
If I were learning how LEDs work, I would not have wasted any time whatsoever on the search results that the author spent a lot of time on. They were obviously (to me) the wrong articles on the face of it, because they were covering aspects that weren't really what I was looking for (the wrong sort of detail and emphasis).
So I think I would have been off and running pretty much immediately with the web search results rather than spending time on the clear dead ends.
ChatGPT gets me there too, after enough back-and-forth, but it takes longer for me to zero in on what I'm looking for.
I say this not to say that ChatGPT is in any way bad for this. I'm just noticing a difference in how the two of us engage in learning new topics. Perhaps the reality is that for some people, ChatGPT is a godsend, and for others, it's fine... but hardly an improvement for this use use case.
It would explain a lot.
(Also, when did learning stop being fun??)
ChatGPT addresses a scalability problem: not everyone has access to a tutor or can just call up a teacher or mentor to learn and ask questions. But some in the tech industry claim that ChatGPT is as good as or even better than human instruction, which to me seems totally off base.
The biggest problem I see in using LLMs as a teacher-substitute is that LLMs answer the questions you ask, whereas a good teacher tells you what you need to hear. Maybe this is solvable with specialized model tuning, but we need to actually solve it before telling kids that the best way to learn is to talk to the computer.
It's also magical when you summarise the understanding you've reached back to it, and it can confirm or tweak it for you.
In other ways it's also nice to just pay for it and then to be in an advertising free space.
One critique is that when you ask it to compare things it's often too balanced or too positive/enthusiastic ("both are great for different reasons!") when what you want is a more sober analysis. But you can usually do some prompt management to adjust it back to a reasonable range.
I have not found these explanations sufficient, while I have done a bit of chemistry and physics I understand the basics of light emissions, but reading this has the same value as reading the wiki article on LEDs to me.
(Edit: - and you get sources right away with wikipedia!)
You can't possibly know that.
Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.
As I said in https://simonwillison.net/2023/Apr/8/llms-break-the-internet...
> This is the thing I worry that people are sleeping on. People who think “these language models lie to you all the time” (which they do) and “they will produce buggy code with security holes”—every single complaint about these things is true, and yet, despite all of that, the productivity benefits you get if you lean into them and say OK, how do I work with something that’s completely unreliable, that invents things, that comes up with APIs that don’t exist… how do I use that to enhance my workflow anyway?
I encourage you to experiment more! Here are my notes so far: https://simonwillison.net/tags/llms/
> There was a boy called Eustace Clarence Scrubb, and he almost deserved it
> Everything starts somewhere, although many physicists disagree.
> It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet
> My father had a face that could stop a clock.
> It is important, when killing a nun, to ensure that you bring an army of sufficient size.
> In the myriadic year of our Lord—the ten thousandth year of the King Undying, the kindly Prince of Death!— Gideon Nav packed her sword, her shoes, and her dirty magazines, and she escaped from the House of the Ninth.
There's just no end to these.
And, of course, science is full of these too, one that jumps to mind is Shinichi Mochizuki's claimed proof of the ABC conjecture which has been proven flawed as it often happens but it was certainly a credible proof despite written in a language no mathematician have ever seen.
Weirdly, LLMs do that too.
Have you heard of the "Let’s be bear or bunny" pattern?
My iPhone hallucinated that the other day, running a LLM directly on the phone!
Yes but those are as every other thing they emit, indeed , "Let’s be bear or bunny", I heard it called AI hallucination I personally like to call it word salad.
> The man on the moon fell off his ladder one Tuesday, a common enough occurrence that nobody really paid it any mind. Truth is, gravity's always been a bit of a show-off, even in the star-dusted emptiness of space.
You can't rely in it being 100% correct, but that's very different from it having no informational value at all. When it comes down to it, you can't rely on anything being 100% correct. I recall finding multiple errors in textbooks in the past, and certainly Wikipedia is wrong about all kinds of things; that doesn't make them useless. It just means in situations where it's critical to be correct, you need to double-check. But often that's not necessary, and when it is, it's a lot easier to start with something and then verify it than to not have the tool in the first place.
Nonsense. The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. They are much, much more likely to be correct, however, so their information value is greater than zero by definition.
What you're claiming is equivalent to "search engines can find incorrect information, so search engines are worthless for information retrieval". Which is bollocks.
Which is the case
> They are much, much more likely to be correct,
They are not. It's just apophenia to find meaning in the word salad.
The irony of people complaining that LLMs confidently assert things that are completely wrong...
References:
Having a conversation allows me to figure out what I’m trying to figure out.
Once I know that, I know what to look for.
To me, the value of the conversation isn’t its perfect accuracy, but the expressiveness and ease of veering in any direction that seems interesting at the moment. The efficiency gains when jumping around within a subject are incredible.
For me it’s consistently been much more helpful than google.
Scary part is I don't know enough about the subject to tell apart truth from falshood because they are stated in exactly the same confident manner. Also most things are true with falsehoods sprinkled in between
On the other hand, I feel like my experience with Phind has been so great because with all dev stuff I ask it it's always been something I could try out right away.
With non-dev stuff that isn't always the case. I'd love an LLM that can do the same thing as Phind, as in providing the sources it's using, while also working for everything.
I'm not taking anything away from Chat-GPT for this use case, but I see the above pattern in almost every Google bashing thread on HN.
* Various new english idioms. I read a lot, but still find new ones
* More optimal translations between languages i'm ~native in.
* ton of new JS, Python libs
* Got deeper in SQL (recursive queries, etc)
* Can use CSS way better than before
* Mermaid
I wrote https://chatcraft.org/ as a friendlier UI for learning programming related things.
But I was bitten by a series of completely made up facts, and my skepticism has forced me to research nearly every complex response since then to confirm.
This fatigue has really shut down my enthusiasm. Maybe I'll research a topic 3-4 hours a week now.
- summarising entire chapters (summarise chapter 8 of book 1)
- asking to verify my own summaries (in summary, smith writes that ... am i correct?)
- explaining certain passages in ever more detail (explain ...; explain in simpler terms; im still confused, explain in more details;)
---
One very interesting case is when GPT produced an error, and I had it go through the error with me. In the end GPT managed to spot its own error and correct itself. Here's an excerpt, edited for brevity:
* gpt: [...] When the demand is too low, it encourages growth; when the demand is too high, it restrains growth. [...]
* me: "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth." is this correct?
* g: Apologies for the confusion in my previous response. The correct statement should be:
When the demand for labor is too low (slow), it encourages higher wages and population growth; when the demand for labor is too high (fast), it restrains population growth by adjusting wages to find the equilibrium between demand and supply.
(GPT still giving the wrong answer.)
* m: i am still confused. can you explain in more detail
* g: [...] Smith argues that the demand for labor regulates population growth by influencing wages. If wages are too low, fewer children will be born, and the population growth will be slow. If wages are too high, more children will be born, and the population growth will be fast. [...]
(At this point GPT was able to give me the correct response.)
* m: so when the demand for labour is low, wages decrease and population growth slows down. when the demand is high, wages increase and population growth speeds up. correct?
* g: Yes, that's correct. [...]
(I pointed out that this was in contradiction with the earlier response.)
* m: did you not contradict yourself? "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth."
* g: Apologies for the confusion in my previous response. I misunderstood your question. Let me clarify:
When the demand for labor is too low, it restrains growth; when the demand is too high, it encourages growth.
So, when the demand for labor is low, wages decrease, and population growth slows down. Conversely, when the demand for labor is high, wages increase, and population growth speeds up. I hope this clears up any confusion.
(I then inserted the original erroneous response as a prompt, and gpt was able to "remember" that it generated the response earlier.)
* m: "When the demand is too low, it encourages growth; when the demand is too high, it restrains growth." what is wrong with this passage?
* g: I apologize for the confusion earlier. There is an error in the passage you provided. It should be:
This demand for labor operates like the demand for any other commodity, regulating the production and growth of the population. When the demand is too low, it restrains growth; when the demand is too high, it encourages growth. This dynamic determines the rate of population growth in different countries, such as rapid growth in North America, slow and gradual growth in Europe, and a stationary population in China.
---
i should mention that i am using the paid version (GPT-4). i tried the free model, but the quality was significantly worse. you can recorrect it multiple times consecutively and every time it would say that it was wrong the last time, and recorrect itself again and again.