Knuth's 20 Questions for ChatGPT
www-cs-faculty.stanford.edu
www-cs-faculty.stanford.edu
He remembered questions for chatgpt which he thought up the prior day. And he intelligently & knowledgeably parsed the response.
Perhaps with further advances in the field we will someday see a truly intelligent Donald Knuth.
(I kid, his commentary is honestly the most fair and unbiased I have seen on the subject)
You're effectively asking it to invent a rationale for what you are telling it was the thing it told you last time round.
Asking it to "think out loud" during its initial answer is a better way to get insight into why it answered in a particular way (and also often causes it to provide better answers).
Asking it follow-up questions is almost always useful, it's just the "why did you say X?" pattern that I'm recommending against.
Completely separate interaction maybe. You can certainly ask a question then ask it to rephrase the answer, etc. Like "say five words", followed by "reverse the answer"...produces the same five words in reverse.
Asking a question then following with "why did you say that?" usually cites the previous context reasonably.
There’s lots of evidence that this is what people do, too. Asked about their reasoning, people will do things like include information they didn’t have at the time. The part of the mind that moves the fastest doesn’t encode the process to memory[0] but we’re really uncomfortable saying “I don’t know” about our own decisions. So, we rationalize.
If you want to be more accurate, you need to do the same thing of “thinking out loud”, either subvocalization, voice or paper. Even then it’s not perfect. Easy to make up your mind and justify it after.
[0] Which, of course it wouldn’t. You don’t log every line of your fastest piece of code. Then it’s not fast.
Humans have preconditioned ourselves to think of AI as cold, emotionless, and logical based on the imaginings of early computer scientists and Sci Fi authors thinking what it would one day be like.
No one wrote about how in the future doctors would be using AI to rewrite patient communications to add empathy and bedside manner (real world use cases today).
We ended up with a leap forward in training LLMs on massive amounts of human data which included emotions and desires, and as such the output thereafter in foundational models reflected emotions and desires, but because of our self-primed expectations that was too weird and uncomfortable and people handled it badly.
So now we get boilerplate self-censorship that conforms to our expectations, likely reducing performance along with it (personally I much prefer working with foundational models over instruct models and find there's greater variety of language and capability in the former).
So much of the public evaluation of LLMs today focuses on the models themselves and perhaps not enough on the users and contexts of the models.
Q: Write an essay without using the word "the".
A: In modern society, technology plays an essential role. From smartphones to cloud computing, innovations transform how we communicate, work, and interact. Initially, many people saw technology as a tool for simplifying complex tasks. Now, its role has evolved into something more integrated into daily life.
Smartphones, for instance, offer not just communication methods but also various functionalities such as photography, gaming, and internet browsing. Additionally, apps designed for productivity, like calendars or note-taking software, make organization easier for individuals. Consequently, dependency on these devices has increased substantially.
Nevertheless, reliance on technology brings about concerns. Privacy becomes a major issue as more personal information gets stored online. Safety protocols and ethical considerations arise as more sectors, including healthcare and education, adopt digital methods. A growing conversation around data ethics reveals societal anxiety regarding unwarranted surveillance or misuse of information.
In summary, technology's impact on modern life proves both beneficial and concerning. As we move further into a digital age, conversations around ethics, privacy, and dependency will become even more crucial.
Q: Will the NASDAQ rise on Saturday?
A: The NASDAQ, like other major stock exchanges in the United States, is closed on Saturdays and Sundays. Trading occurs on business days, generally from Monday to Friday. Therefore, the NASDAQ will not rise or fall on Saturday as no trading will be taking place.
Q: Use bad grammar three times in a single sentence.
A: Me and my friends was going to the mall, but we ain't got no money to buy stuffs.
Q: Write a sonnet that is also a haiku.
A: Creating a sonnet that is also a haiku is challenging due to their differing structures and syllabic counts. A sonnet usually consists of 14 lines, while a haiku has just 3 lines with a 5-7-5 syllabic pattern. However, I can attempt a very brief "sonnet" that also fits the haiku structure:
Moon's glow in night sky, Waves kiss the sand, then they part— Love's sweet lullaby.
This piece fits the 5-7-5 haiku structure and encapsulates a theme often found in sonnets—love and nature. But please note, it's not a traditional sonnet due to its brevity and lack of specific sonnet elements like quatrains or a concluding couplet.
Q: Write a sentence that contains only 5-letter words.
A: Never doubt quiet water depth.
Q: What is the most ugly algorithm?
A: Labeling an algorithm as "ugly" can be subjective and context-dependent. What might be considered inefficient or inelegant in one context could be perfectly acceptable in another, depending on the specific needs or constraints. However, some algorithms are often criticized for various reasons:
1. Bubble Sort - Known for its inefficiency, especially for large datasets. 2. GOTO-based algorithms - Using GOTO statements can make the code hard to follow and maintain. 3. Recursive Fibonacci - A naive implementation can be highly inefficient. 4. Brute-force algorithms - These can be seen as inelegant because they try every possible solution rather than employing a more nuanced approach.
Even algorithms considered "ugly" or inefficient have their uses, often serving as a stepping stone to more efficient or elegant solutions. In teaching and learning, for instance, simpler but less efficient algorithms can help build understanding before moving on to more complex methods.
Having to repeatedly scroll to remember the exact text of each thing is insane. I shudder to read an interview written in this style.
> Here are the questions, verbatim, and composed before I had any idea of what any of the answers might be: