lsedgwick

Fair, you're probably right in this case.

Please don't paste ChatGPT or BARD answers as HN comments in general? In this specific case, no, LLMs don't reliably know about themselves. They're trained on a big corpus of internet text, then trained by rough reinforcement learning to say and not say certain things about themselves, then given a little more information about themselves in system prompts.

Do you think it's true? I got my hands on the pdf see no trace of it in table of contents or text searches.

You're right about the basic mistakes they can make - they can also excel at the same tasks if prompted differently. But I was making a slightly different point. The point is they can reason about things in a better-than-chance way (I mean, a much-better-than-chance way) when given problems not in the training set. Have you read the Codex paper? Seriously, go look, an LLM even from years ago (which is like decades ago in ML-time) is often able to write code to solve novel programming problems that were handwritten to not be in the training set! Also, the benchmarks against which GPT3, InstructGPT, and GPT4 were tested are illuminating about its problem solving ability on novel problems, these are described in the three respective papers.

Secondly, quality is highly variant and there are traps the context window can fall into which causes especially bad results. Jeremy Howard has a great video (https://www.youtube.com/watch?v=jkrNMKz9pWU, starting at 18:05 the 'limitations and capabilities' section is only 13 minutes long) talking about how quality depends on: how you frame your prompts, model power (4 does a lot of stuff that 3.5 can't), and whether you're in a kind of "context trap" of repeated mistakes.

Of course, some people like to point out that if it's so "finicky" and variant, it is "dumb." Sure, if you like. I'm not interested in whatever definitions you're using those things, the objective and observable point is that given well-known prompting practices, LLMs can do something functionally equivalent to reasoning about novel problems, and more powerful ones can reason about more powerful and difficult things.

I re-phrased your prompt (instead of "prove a false thing" I made it like "decide whether this thing can exist, and prove your answer"). And added a little well-known boilerplate prompt sugar. It seems to have done a better job.

https://chat.openai.com/share/53214f0c-17f7-4a3d-95be-8fd676...

There are certainly emergent capabilities that come out of the training process of being able to predict next tokens, and it's shown that LLMs often can solve logic and math problems, riddles, etc. which are not in it's training data because deep in the internal layers of the model it has something resembling logic and reasoning abilities. This can be seen in the "benchmark" results of papers like the GPT-3 and InstructGPT papers, among many others, where they tried pretty hard to test the models against questions which were not part of the training.

This is also shown in the Codex paper, where they trained an LLM to write code and then watched it solve a number of code problems they handwrote originally to make sure the problems could not have been in the training data.

Try it out yourself, make up some little math word problems and ask chatGPT or something.

Of course, advent of code will be much more challenging problems, but to get help with some subcomponents of the problem a motivated participant would likely try to use the most recent, powerful, and advanced models which outperform the results from papers written a few years ago, and outperform the free chatGPT.

I think the right context for thinking about the quoted advice is for a person's education. If true beginners didn't ever handwrite a recursion algorithm, a list sorting algorithm, a CRUD application, an assembly interpreter, a neural net forward pass, etc., because they copied the code from an LLM, would they benefit from that? And if it's true for beginners, is there ever a point in one's learning journey where it stops being true and we don't have to learn or practice anything anymore which can be solved by an LLM, or never have to understand new concepts and tools even though the LLM can work with those concepts and tools for us?

I think there's a both-and answer for the contest. Maybe have one competition where the unenforceable spirit of it is, don't use LLMs for help, and another one where the challenges are just made... quite harder... so that even people who use the LLMs still need to marshal great ingenuity to use them better than others (e.g. the #1 spot is someone who used RAG and chain-of-thought better than the #2 spot, and also had better intuition of what to trust vs challenge from the LLM outputs).

Emissions tests are a poverty tax that play a strong role in punishing people trying to buy affordable old cars and either pushing the demand curve for new cars, or pushing poor people into illegality or carlessness.

hey man, please shoot me an email, I've not quite been in your shoes but would love to talk it out anonymously if you like, or at least just listen if you want to vent more. It's (hidden behind icloud for obvious reasons) chokers_blatant0o@icloud.com

as many others now have said, you have great value (in my worldview, you have infinite value as a divinely created being, for that matter, not that I expect you to just believe me when I say that). Losing or never attaining success, money, and glamour does put people into disfavor with much of the world, but when you lose those things I really think it can be an opportunity to look within yourself and discover value that comes from a wholly different frame of reference. I’ve met enough people without great charisma, money, beauty, or power, who have still found contentment and joy (at times, at least; many still go through up and down seasons of mental struggle on a monthly or hourly cycle!) because they have found their own worth in the light of different schemas of valuation. And other people recognize these figures too! People know when they meet someone who is content to authentically be themselves without the esteem that comes from, say, a good job, or anything else. Don’t you sense it sometimes, when you meet someone like that? It’s not just charisma, it’s something better.

pardon the change in topic but what does GP stand for? is is like OP but for a comment thread?

I'm truly sympathetic towards the people affected, especially visa holders. But speaking solely of the macro effects, I have to say, I'm happy this day, stretching from last year to now, has come. We as a community were a very annoying group of people when we were so damn rich and when we thought we were essentially untouchable. And count me in that as well! But a little awareness of layoffs here and (for some people) a little fear of ChatGPT replacement there... well that's a grounding experience. Not to mention the number of people who had sunk into so much ennui and depression over how boring and easy it was. Now a lot of us are jobless and can feel some fire in our bellies again, a time to fight or die.

Reminds me of Alan Turing's wildly undernoted remarks about ESP in his seminal essay "Computing Machinery and Intelligence"

> I assume that the reader is familiar with the idea of extrasensory perception, and the meaning of the four items of it, viz., telepathy, clairvoyance, precognition and psychokinesis. These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them! Unfortunately the statistical evidence, at least for telepathy, is overwhelming. It is very difficult to rearrange one's ideas so as to fit these new facts in.

(This is from a paragraph dealing with ESP's implications are for the existence of thinking machines.)

https://redirect.cs.umbc.edu/courses/471/papers/turing.pdf

I guess this makes me a little demoralized about the utility of "anti-big-tech government actions," because to the extent any of these actions succeed, it's probably just sending a clear message to enhance state influence/access of other tech companies. If I get excited (and I want to) because "Amazon might be broken up!!1!", maybe that just means Google finally got the message and decided to put backdoors into <insert platform here> on behalf of <insert agency>.

I think this demonstrates a way in which all centralized power becomes, for good or for ill, partially absorbed into state power. A major country's government will always be able to exert lots of leverage against, e.g. tech companies, whether for legitimate or illegitimate reasons, it matters not which. They can always threaten regulatory or legislative actions which would be annoying or devastating to shareholders. Hence, behind-the-scenes conversations will always have an implicit fact of "please censor as we ask, provide data and backdoored access to which servers we ask, please don't take on the wrong political causes, elsewise there will be consequences."

Goodness knows that if the intelligence community or state law enforcement has ever wanted access to anything in AWS, now of all times would be when they have the easiest time asking for it! Anything to curry favor with someone who can speak a word of support to federal agencies and state officials.

Not to say companies should or shouldn't RTO etc, but the shocking freedom and flexibility of remote work in tech is such a gigantic class marker and insane lifestyle luxury that I'm surprised it's not more often identified a such.

How few people in history have been able to travel to exotic places or visit far-off friends and family, while still making decent money, and do this almost as much as they want (supposing lifestyle and relationships that allow this, which is true for many people). I mean, people in tech who vacation in tropical climes then spend cherished time with parents and siblings and friends living thousands of miles away, and don't worry about running out their vacation time: dozens of weeks of cherished traveling, if that's what you want! Along the way developing fundamentally different types of family relationships or accruing worldly sophistication.

Forget about little medical or technological things that mark a break from the past: lots of people can afford tylenol and iPhones and Spotify. This is a class marker of epic proportion. This is a way of life that Tolstoy ascribed to the very wealthiest of Moscow's aristocratic families and bachelors (most certainly not even all of the aristocratic class).

Reminder that our regulatory apparatus allows these people to have phone numbers and make and receive phone calls without being able to be held accountable.

They're saying these were notes for a speech for a "communcations class" where he was cosplaying Gorden Gecko and not presenting his true beliefs? That so hard to believe. These notes could not more obviously be an internal memo, filled with clear reasoning about balancing the needs and visions of different internal departments, meant to persuade and stimulate conversation about Google's strategy. I don't see how this "it was for a speech and I was being hyperbolic on purpose" thing isn't getting laughed out of the water.

This is a common failure mode tech people get into in debates: finding an extreme example as an analogy. There's lots of obvious differences of context and differences of degree which get swept over by a very abstract kind of stripped-bare framework, "what's the essential philosophical difference here?" If someone can't see those differences it's almost pointless trying to enumerate them because honestly it feels like a disingenuous debate trick (my apologies if I'm wrong in that accusation).

You have an extremely wealthy corporation incentivized to collect massive amounts of very personal data about every part of you and you desires and psychology, incentivized and well-able to collect it in hidden and powerful ways across the web and build deep learning models of your psychology and desires, and store them in a place where it's possible for nefarious actors and governments to either legally or illegally access and use that information for unknown purposes. If you object to this characterization, fine, start with the objection, but it's an obvious characterization which I've presented, and it's obvious that this is what people are concerned about! Starting the conversation with "it's no different, philosophically, from your grocer knowing your love of chocolate" is such a non-starter only because it seems so disingenuous, like, how could you not see how your interlocutor sees it differently? It just presents so clean and abstract a little simplification, though, so it's handy for winning point in a debate setting.

Either the site really doesn't support https (odd, since it is the very page that supplies the verification hashes!) or something more scary is actually happening and it's getting universally man-in-the-middled, which feels plausible for a site that offers cryptographic tools for download.

To add on with another question, are there systems (like password managers, or others) which have "double password" as a first-class feature? For instance, a hacky way could be if personA knows passwordA only, and personB knows passwordB, and the literal password for a system is the concatenation "passwordA + passwordB" - you could get that if both people sat at the same keyboard (or did something else annoying), but a password manager for instance would need first-class support for that feature to be able to have the two individuals launch a shared session to enter both passwords. Or I would even love a system where if at least 2 out of 3 people entered their passwords it launched a shared session: no one single point of failure either for compromising a person or for that person getting hit by a bus.

> Thomas Tymoczko has pointed out that there is nothing special about eight "buffalos";[2] any sentence consisting solely of the word "buffalo" repeated any number of times is grammatically correct.

My favorite part of this. Repeated 10 times, for instance, could mean "NY bison NY bison bully bully bison NY bison bully." (The New York bison who are bullied by other NY bison, well they themselves bully some bison whom other NY bison bully.) Any number of times unfurls out of, like, kind of a context-free grammar replacement scheme from a base case:

  "bully bison" -> base case
  "* bison *" -> "* bison bison bully *"
  "* bison *" -> "* NY bison *"

fair. This was similarly amusing at least: > ERNIE’s opinions are surprising, to say the least. It believes the best American president is Richard Nixon:

Impressive to see the RLHF penetrate deep enough into concept maps to teach the model (surely implicitly!) that the best US President has to be whichever one normalized relations with China.

> Bizarrely, this happens even when the question is completely unrelated to China: you get the same error message when you ask, “Why is Hawaii a part of the US?”

Not bizarre at all, reveals they probably fine-tuned / RLHF'd on not answering a bunch of things that look similar, like, a bunch of variations of the question about Taiwan.