Don't believe ChatGPT – we do not offer a "phone lookup" service

uh, you mean stalker / scammer platform? This would be a major privacy violation.

It’s an unprincipled hack, a bizarre dependency to add to your project, it probably feels like admitting defeat to the all powerful AI… but it does 90%-solve the problem.

For example - https://platform.openai.com/playground/p/default-translate-c...

The codex models are intended for doing work with code rather than language and may give better results in that context. https://help.openai.com/en/articles/6195637-getting-started-...

It may be more helpful to look for better answers on Amazon's help pages for SNS and AWS SDK.

I'm not super familiar with ChatGPT internals, but there are plenty of ways to tack on uncertainty estimates to predictions of typical "large scale ML models" without touching Bayesian stuff (which only work for small scale academics problems). You can do simple parametric posteriors estimation or if all you have is infinite compute and don't even want to bother with anything "mathy", bootstrapping is the "scalable / easy" solution.

you could cross check this stuff too with yet more models.

Whether or not the information that comes back from those searches is reliable is a whole other question.

I would love to learn what the latest research is into "factual correctness" detection. Presumably there are teams out there trying to solve that one?

Yes. It's a very active area of research. For example:

Discovering Latent Knowledge in Language Models Without Supervision (https://arxiv.org/abs/2212.03827) shows an unsupervised approach for probing a LLM to discover things it thinks are facts

Locating and Editing Factual Associations in GPT (https://arxiv.org/pdf/2202.05262.pdf) shows an approach to editing a LLM to edit facts.

Language Models as Knowledge Bases? (https://aclanthology.org/D19-1250.pdf) is some slightly older work exploring how well LLMs store factual information itself.

Yann Lecun has posted a lot recently about this but basically LLMs are a "useful offramp on the road to AGI".

At its core using an LM alone to solve factual problems seems silly: It's not unlike asking Dall-E to draw DOT compliant road signs.

I've gone at length at how unfortunate it would be if LMs start to get a bad rap because they're being shoehorned into being "Ask Jeeves 2.0" when they could be so much more.

Personally, I use ChatGPT (the paid version) and Copilot every day and find them awesome enhancers.

The rich keep getting richer.

I promise you most people do not know this.

This is an example of a whole range of beliefs about LLMs that are very common (even in the field itself), because they were obviously true for small models, but that might not necessarily hold for larger models. There's a lot that we don't know about LLMs, but we do know that they exhibit emergent behaviors as they scale. Smaller models don't really have world models, just language models, but these larger models have started developing clear world models once given the capacity and data to do so.

As for the existence of a concept of factuality, I found this paper[1] very interesting. It details an unsupervised method to identify which internal activations of the model correspond to factual statements, regardless of what the model ends up saying. Looking at those internal activations rather than just the model's output even reduces the model's susceptibility to prompts that lead it towards saying the wrong answer.

[1] https://arxiv.org/abs/2212.03827

Specific example: "why do we have first-person subjective experiences? List current theories. For every theory, assign a truthiness float 0..1, where 0 means you're sure it is wrong, and 1 means you're absolutely sure it is true"

From experimenting with this, it will shift the output, sometimes drastically so, as the model now has to reason about it's own certainty; it tends to make significantly less shit up (for example, the non-truth-marked version of the output for the query above also listed panpsychism; whereas the truth-marked version listed only scientific hypotheses).

So the model _can_ reason about it's certainty, and truth-value; and I strongly suspect it was just not rewarded during RLHF for omitting things it knew to be false -basically, percolating the social lies people tell to eachother- which seems to show up in coding as well.

Edit: see https://twitter.com/sdrinf/status/1629084909422931969 for results

I wouldn't hold my breath. The whole idea of statistical language modelling (much more ancient than Transformer-trained large language models, btw) is to represent structure without having to represent meaning, because we have no idea how to represent meaning. Or, seen another way, we know how to represent structure, but not how to represent meaning, so let's focus on structure and cross our fingers that meaning will naturally sort of emerge, when it feels like it.

So far, we got structure down pat (it's been a few years now, or quite a few, depending on how you see it) but meaning is nowhere to be seen.

Nevertheless, this is an interesting scientific result: one can have smooth, grammatically correct linguistic structure without meaning. Progress has been achieved (and no, this is not sarcasm).

Or it marks the beginning of the next "AI Winter."

> but it clearly shows that ChatGPT can funnel potential customers towards a product or service.

And the next logical step is "chatgpt keywords advertising." Which is right back where we started.

> does 2 pounds of feathers weigh more than 1 pound of bricks?

> No, two pounds of feathers weigh the same as one pound of bricks. The weight of an object is determined by its mass, and one pound of any object weighs the same as one pound of any other object. However, feathers are less dense than bricks, so two pounds of feathers will take up more space or have a larger volume than one pound of bricks.

Huh. It's interesting that some of these "wrong paths" are quite so repeatable. Wonder if that means there's also a way to band-aid them when found, the way a human might be corrected for a misconception.

Agreed. But then it begs the question: what purpose does ChatGPT serve (other than for entertainment purposes or cheating on your HS/college exam)? If you have to verify its information by other means, then you're not really saving much effort.

It’s worse than that. It’s wrong, you cannot correct it and it makes up supporting citations on the fly. Very few humans behave like that.

Even worse because it has no clue when it might be completely wrong and yet it will be confident in its answer.

ChatGPT can only bullshit

Could someone push "wrong" opinion heavily online to sway the opinion of AI?

I can only imagine a bot that learned from 4chan.

Providing detailed information on the usage of a service that has never existed is a brand new kind of incorrect that is carelessly causing the rest of us grief.

Even if it’s wrong, dangerous, misleading, fundamentally flawed as a concept whatever. Big tech and money will find ways to keep putting it in front of us.

It correctly stated that they are, but when it went on to prove that this was the case, it generated a table of the frequencies of the individual letters in these two words, and the table looked like this.

    Letter | Frequency in | Frequency in
           | “pannekake”  | “kannepake”
    - - - - - - - - - - - - - - - - - - -
    a      | 2            | 2
    e      | 2            | 2
    k      | 2            | 2
    n      | 2            | 2
    p      | 2            | 2

I think that’s a bit pedantic and not very helpful… I’m not typing this comment, my brain is just sending signals to my hands which causes them into input data into a device that displays pixels that look like a comment

I mean, you could say that about a person too, as you don't know how much that they are saying is bullshit.

For one, you are technically correct about ChatGPT not recommending. It cannot perform such action. On the other hand, from the POV of the questioner, it's hard not to feel being recommended something when you ask "What do you recommend" and it says "I recommend that...". You are, for some intents and purposes, being recommended something at that point.

If you are a mobile network operator.

Or, you can convince people to install something on their phone that sends you their location along with their phone number.

If a phone number is for a mobile phone then looking up the location doesn't make sense at all: mobile phones are mobile.

I guess you could try and crawl an index of business phone numbers and associate those with the listed address for businesses, but that's a completely different business from running a geocoder.

You could provide a bit of geographical information about the first three digits of a US phone number. I imagine that's not what users are actually looking for though.

you mean like scammers and stalkers? (ok, and probably Meta)

Bonus points for using ChatGPT to implement this end-to-end.

We do have python tutorials and SDKs showing how to use our service for ... geocoding, the actual service we provide.

I wrote the post mainly to have a page I can point people to when they ask why "it isn't working". Rather than take the user through a tour of past posts I need something simple they will hopefully read. But fair point, I can add a link to last year's post about the erronious youtube tutorials as well.

What I think you can't appeciate is the difference of scale. A faulty youtube video drives a few users. In the last weeks ChatGPT is sending us several orders of magnitude more frustrated sign-ups.

It may be an old problem, but I guess users are more use to a random YouTube video with wrong information. But the computer is always right so ChatGPT is always right, so users may be more annoyed to discover that the recommendation is wrong and blame them instead of ChatGPT.

This is a service that OpenCage provides, and for whatever reason OpenCage happens to be one of the popular services for this use case. (Maybe it’s because you get the text description of location back right away without having to do a round trip through a heavyweight on-screen map, maybe their free tier allows more requests than most, maybe their api is easier to use, maybe they are lucky or skilled with SEO and their tutorial happens to be the first result for some common phrases, who knows.)

So there’s this process that starts with a search for “convert phone location to address”, often involves the OpenCage api, and ends with a happy developer getting the information they wanted. Various algorithms pick up on the existence and repeated traversal of this happy path.

In another part of the internet, code tutorial content farms notice a demand for determining an incoming call’s location from the number that’s calling. They search for things like “convert phone number to location” and “convert phone number to address”. Some of these searches end up falling into the nearby well-trodden path of “convert phone location to address” and the content farmer is presented with the OpenCage api. They mess around with the api for a bit and find they can start from a phone number and get a successful api call that returns a lat/lon pair. A successful api call that returns legitimate-looking lat/lon data is all they need to make a video, they make it and post it. Higher-quality, more scrupulous code tutorials attempt to answer this same demand but find it’s not possible, so those tutorials don’t get made, leaving the less scrupulous ones that stop with a successful-looking api call to flourish in this space. The tutorial is doing well, so the content farms endlessly recycle it into blogspam.

As a result, OpenCage starts getting weird usage patterns, tracks them down, finds the source is these tutorials, and makes a post about it.

Some time later, ChatGPT is released. People are astounded with its ability to write code and start using it for this purpose. Naturally, some of those people have the same demand as the previous generation of devs who stumbled onto the unscrupulous code tutorials. Because of the blogspam, ChatGPT’s training data includes many variations on the tutorial, and just as naturally it ends up reproducing that tutorial when asked - except ChatGPT’s magic kicks in and instead of including (what its embeddings see as) some weird unrelated area-code-to-string nonsense from the tutorial, it just bullshits some plausible-sounding data plumbing code instead. Unfortunately, because the tutorial never worked in the first place, that weird hacky irrelevant bit that ChatGPT ignored happened to be the secret sauce that makes the whole thing superficially appear to work.

As a result, OpenCage starts getting weird usage patterns, tracks them down, finds the source is ChatGPT, and makes a post about it.

In deference to Hacker News’ policy of keeping comments pleasant, I will elide the analysis of the process that leads to comments accusing OpenCage of nefariously engineering the whole thing for attention.