Anthropic Claude 3.5 can create icalendar files, so I did this

This is semi-offtopic, but "trust but verify" is an oxymoron. Trusting something means I don't have to verify whether it's correct (I trust that it is), so the saying, in the end, is "don't verify but verify".

> why not just start there?

Because there are many categories of problems where it's much easier to verify a solution than it is to come up with it. This is true in computer science, but also more generally. Having an LLM restructure a document as a table means you have to proofread it, but it may be less tedious than doing it yourself.

I agree that asking straightforward factual questions isn't one of those cases much like I agree with most of your post.

Off topic, but a funny thing about asking about George Washington's birthday is there are two possible answers because of British calendar reform in 1750 (although we've settled on recognizing the new-style date as his birthday).

footnote [a] on wikipedia: https://en.wikipedia.org/wiki/George_Washington#cite_note-3

This is why I don't use code LLM code generators for citing or outputting a solution that includes the current code, because it's inclined to remove parts that it thinks don't matter, but rather matter a lot further down the line. And if it's not caught in code reviews, that can cause severe and difficult to debug issues. I'm sure there will be an epidemic of these issues in a few years, because developers are definitely lazy enough to rely on it.

yea, I seen it alot on social media where people use ChatGPT as a source for things it possible cant know. Often with leading questions.

Already quite a while ago I was entertained by a particular British tabloid article, which had been "AI edited". Basically the article was partially correct, but then it went badly wrong because the subject of the article was about recent political events that had happened some years after the point where LLM's training data ended. Because of this, the article contained several AI-generated contextual statements about state of the world that had been true two years ago, but not anymore.

They quietly fixed the article only after I pointed its flaws out to them. I hope more serious journalists don't trust AI so blindly.

I believe it’s going to become counter productive sooner than anyone might think, and in fairly frustrating ways. I can see a class of programmers trading their affinity with the skill for a structurally unstable crutch.

I was using Perplexity with Claude 3.5 and asked it how I would achieve some task with langchain and it gleefully spat out some code examples and explanations. It turns out they were all completely fabricated (easy to tell because I had the docs open and none of the functions it referred to existed), and when asked to clarify it just replied “yeah this is just how I imagine it would work.”

Agree with this sentiment, erosion of trust and potential issues. The illusion of facts and knowledge is a great moral hazard that AI companies are willing to step around while the market share battles play out. More responsible AI companies, stronger government policy, better engineering and less dumb users are all part of the solution here.

This is more solvable from an engineering perspective if we don't take the approach that LLMs are a hammer and everything is a nail. The solution I think is along the lines of breaking the issue down into 2-3 problems: 1) Understand the intent of question, 2) Validating the data in resultset and 3) provide a signal to the user of the measure to which the result matches the intent of the intention.

LLMs work great to understand the intent of the request; To me this is the magic of LLM - when I ask, it understands what I'm looking for as opposed to google has no idea, here's a bunch of blue links - you go figure it out.

However, more validation of results is required. Before answers are returned, I want the result validated with a trusted source. Trust is a hard problem..and probably not in the purview of the LLM to solve. Trust means different things in different contexts. You trust a friend because they understand your worldview and they have your best interest in mind. Does an LLM do this? You trust a business because they have consistently delivered valuable services to their customers, leveraging proprietary, up-to-date knowledge acquired through their operations, which rely on having the latest and most accurate information as a competitive advantage. Descartes stores this mornings garbage truck routes for Boise IA in its route planning software - thats the only source I trust for Boise IA garbage truck routes. This, I believe is the purpose for tools, agents and function calling in LLMs, and APIs from Descartes.

But this trust needs to be signaled to the user in the LLM response. Some measure of the original intent against the quality of the response needs to be given back to the user so that its not just an illusion of the facts and knowledge, but a verified response that the user can critically evaluate as to whether it matches their intent.

Maybe it will be analogous to steel. For most of the post-nuclear age, steel has been contaminated with radionuclides from atmospheric nuclear weapon use and testing. To get "low background" steel you had to recycle steel that was made before 1945. Maybe to fact-check information we'll eventually have to go to textbooks or online archives that were produced before 2023.

(Steel contamination has slowly become less of an issue as most of the fallout elements have decayed by now. Maybe LLMs will get better and eventually the hallucinated "facts" will get weeded out. Or maybe we'll have an occasional AI "Chernobyl" that will screw everything up again for a while.)

> The "off by one" predilection of LLMs is going to lead to this massive erosion of trust in whatever "Truth" is supposed to be, and it's terrifying and going to make for a bumpy couple of years.

This sounds like searching for truth is a bad thing, but instead is what has triggered every philosophical enquiry in history.

I'm quiet bullish, and think that LLMs will lead to a Renaissance in the concept of truth. Similar to what Wittgenstein did, Plato's cavern or late middle age empiricists.

Having also been burned by "better to check Wikipedia first" hallucinations, I find the Anthropic footnotes system to be essential. The "confidence and bravado" is indeed alluring at first, but eventually leads away from using LLMs as search engines.

You fear that over time, artificially intelligent systems will suffer from increasingly harmful variance due to deteriorating confidence in training data, until there is total model collapse or otherwise systemic real-world harm.

Personally, I believe we will eventually discover mathematical structures which can reliably extract objective truth, a sieve, at least in terms of internal consistency and relationships between objects.

It's unclear philosophically whether this is actually possible, but it might be possible within specific constraints. For example, we could solve accuracy of dates specifically through some sort of automated reference-checking system, and possibly generalize the approach to entire classes of problems. We might even be able to encode this behavior directly into a model. Dates, at least, are purely empirical data, so there is an objective moment or statistically likely range of time which is recorded somewhere, so the problem becomes one of locating, identifying and analyzing the correct sources for a given piece of information, and having a sound-proof method of checking the work, likely not through an LLM.

I think we will come to discover that LLMs/transformers are a highly generalizable and integral component of a complete artificial brain, but we will soon uncover better metamodels which exhibit true executive functioning and self-referential loops which allow them to be trusted for critical tasks, leveraging transformer architecture for tasks which benefit from it, while employing redundancy and other techniques to increase confidence.

Sounds like the burning of the library at Alexandria, but worse.

They aren't the same as humans. They definitely work differently.

Also they've been trained to say something as plausible as possible. If it happens to be true, then that's great because it's extra plausible. If it's not true, no big deal.

While I have worked with one awful human in the past who was like that, most thankfully aren't!

The difference is that you're asking it to perform one intellectual task (write a program) instead of 100 menial tasks (parse a file). To the LLM the two are the same level of complexity, so performing less work means less possibility of error.

Also, the LLM is more likely to fail spectacularly by hallucinating APIs when writing a script, and more likely to fail subtly on parsing tasks.

All tests have this problem. We still write them for the same reasons we do double-entry bookkeeping.

Or three or four or five! https://openreview.net/pdf?id=zj7YuTE4t8

Yes, you can use AI to spot errors in AI output. Done before with good results, but it requires to run 2 different, but equally good, AI models in parallel, which is way more expensive than 1 model.

The problem is that people could be impressed and use it for things where 0.01% could lead to people getting hurt or even get killed.

Yes, just rewording is enough in this case. It will fail if it tries to be clever with the code interpreter as mentioned.

I used:

"Extract the important dates and times from this pdf in Swedish"

[..]

"Great now generate the content of an ics formatted file, just the text, with the schedule above"

[..]

"Now use the code interpreter to create a file with the above content then save the file as an .ics extension"

Jay-Z? Or Bob Dylan?

You wouldn't have them bunched up if you were celebrating the anniversaries, unless all the music you were interested in was from the same year (or only years where 11 September was a Tuesday, say).

Still happens, but it’s Friday now! https://en.wikipedia.org/wiki/Global_Release_Day

Weirdly enough, I always think that 9/11 was a Friday; and now Album day is Friday!

Yup, we tried scraping IMDB for dates too to compare which method was easiest/best and it was spot on (and easier!). I guess the release dates are mentioned enough around the web that it has enough reference. It does use the US release dates for everything though which was one issue, but that’s almost always the first release for major movies anyway.

This is not either/or.

We can have both.

Any organization in 2024 creating a calendar that impacts 500-1000 people should have the awareness that people use digital calendars / apps too.

They should make an effort to publish both PDF and ical.

Since that has yet to catch on in many places -- including the HR departments of large enterprises that publish holiday calendars and the like -- these alternatives ideas are explored.

Once we have the right tools ...

It just takes one tech-savvy parent to do this 10 minute exercise and publish an online calendar that other busy parents can subscribe to.

I used their paid API and the open-webui frontend. They say that information sent through their API is not used for training, unlike the official free chatgpt interface. As a paid service however it's been insanely cheap compared to the time it's saved me.

That's pretty good already for a model that isn't specialized or finetuned. I'm sure things will get better.

I've felt this for a long time, been using Claude for over a year now internally - super impressed w/ it from the start and the 3.5 model is blazingly fast. Which version of C-GPT did you compare it to?

I pay for both Claude and ChatGPT.

Claude 3.5 is now my daily driver but it still refuses to answer certain questions especially about geopolitics so I still have to go back to ChatGPT.

I don’t agree that Claude is insanely superior to ChatGPT though. It still has trouble with LaTeX and optimization formulations. Its coding abilities are good but sometimes ChatGPT does better.

This is why I keep both subscriptions. At $40/math I get the best of both worlds.

Claude’s recent code preview rocks, especially for basic web design (well, not so basic - it was able to design 3d animated svg of winamp).

ChatGPT’s code interpreter and data analysis is killer too.

Seems I’ll need to keep paying for both.

that was still a lame advertisement.

Also, you too should be rotating accounts every few months to prevent doxing. that's just common knowledge and online hygiene.