Britain to use "AI" to answer taxpayer's letters
telegraph.co.uk
telegraph.co.uk
They were going to stop in 2016, but apparently Matt Hancock insisted they continue using dead baby cow skin? (I suspect it might be more complex than that).
In contrast, the hardbound, acid-free, books from my undergraduate days are falling apart.
My brother has some historical documents from our family history that I got a chance to look at last time I was in the UK, one of which is a will from 1872. Looks like it's hand-inked, with extra pencil marks. I have no reason to suspect it's been maintained under exceptionally carefully controlled conditions, normal domestic conditions are much more likely. And it seems fine.
Similar for the family multi-volume book series on, IIRC, world history; the final volume in the series was hastily added, because it was about the Napoleonic Wars which had only just happened. (I don't know what happened to those books, mum didn't want to keep them when dad died).
Also:
1) this is for all laws, not just special ones — I can understand that at some point the UK government will pass a law that people might like to coo over the physrep of in a museum in 2524 A.D., but it's not likely to be the text of "High Speed Rail (Crewe - Manchester) Bill": https://bills.parliament.uk/bills/3094
2) We used vellum for Magna Carta back in the day, because we didn't have anything better to write on. The actual information content today is recorded and transcribed, shared on the web. How long will the web last? For as long as people care to maintain the records.
The future won't get to see 'interesting mistakes' because such would be destroyed as incorrect representations of the thing parliament debated. Even the physicality of the documents won't tell the future generations about the people who lived today, because vellum is now just a weird thing nobody else does.
Printing these things on vellum is creating an artefact for no other purpose than to have an artefact — we may as well carve them into stone if the point is to have longevity.
I think I just said random words until it put me through to some departmen and from there they had a normal call tree via which I got an unrelated human who could tell me who I actually needed to ask for. But I'm not looking forward to the day that no humans are in the loop and unanticipated circumstances are completely unresolvable.
I fear our AI future not because of evil but because of bureaucrats.
Fair, though the advantage of an actual LLM here is that it's not limited to a dumb hard-coded menu, so if done right (I know, I know) an LLM would help a lot.
(One of the disadvantages is that current models sometimes extemporise answers even if none exist).
Though given humans also do so, perhaps warranted in this case.
I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM. Anyone with a phd in machine learning would probably agree.
Ironic demonstration that humans also do what is deemed "hallucinations" when AI do it: https://en.wikipedia.org/wiki/AI_effect
LLMs, transformer models in particular, are artificial neural networks. They have always been AI. AI is the field which led to this, the research is published as AI research.
It's amazing how often us humans (me included!) don't use RAG (retrieval augmented generation) in the form of a search engine and just trust our gut instinct for off-the-cuff responses :D
> Large language models have no formal reasoning, they have no long term recall, they contain no structured logic.
> I’m not saying it can’t be solved. I’m saying it can’t be solved INSIDE the LLM.
Do you mean transformers then? Because that is the current vogue architecture for large language models which is clearly a broader category.
The full details for the current best models are secret, but they're still large language models, and they're demonstrating surprisingly high performance on logic and reasoning.
Now, layer a few sanity checks on top of an LLM, especially some clever thing we haven't invented yet, and I'll totally believe it - the task is absolutely doable, I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources.
But even Transformers really are not just preditive text.
IIRC the original Google usage, Attention is All You Need era, was for translation; and while I would indeed characterise the first few OpenAI/GPT models as "autocomplete on steroids", that changed with InstructGPT, which was the first time I saw them transforming requests into actions, in the form of creating a very simple web game.
> I'd just find it really weird if a predictive engine could do it 100% accurately, using only modern resources.
I currently think there is no such thing as "knowledge" in reality, that such a state is as unrealisable as counting to infinity, that all we can really have are beliefs of varying certainty; in this regard, 100% can never happen in any system including humans — but also, I wouldn't say an AI is "hallucinating" if the error rate was similar to that of a human.
Likewise, I find it really weird how a neural network with the complexity of a mid-sized rodent is able to transform prompts in the most used languages into mostly-correct source code in most programming languages — this is not a thing I would have expected, given the observable lack of employment opportunities for rodents* in software engineering departments.
I could be wrong about both, of course.
* other than furries, who are everywhere ;)
Any company that want's to use an LLM to do "customer service" needs to give it full access to accounts and systems, otherwise I fail to see how it's actually doing to make ANY difference, other than pissing people off. Now I don't advise you to do this, because that's stupid and dangerous, but if you don't it's basically just a search engine with a better query interface. But it fails even at that, remember the Canadian airline where the chatbot just straight up lies?
MTPE, "Machine Translation Post Editing", is what has become the norm.
AI generates your first draft. Humans post-edit the output as a final draft.
I imagine most AI use cases will still have a human in the loop for quality assurance. (The goal of AI doesn't need to be 100% accurate as long as the first draft is able to be post-edited and reviewed by a human who ultimately takes responsibility for the output - assuming post-editing/QA takes less time than writing the first draft yourself)
I think that simple rubber-stamping would happen in any situation where the input was 'good enough' most of the time. And so the Bad Things and hallucinations would still get through
It all comes down to quality at the end of the day. The person doing the work will be fired if the quality of their output isn’t up to standard, assisted by AI or not. That’s how the language translation industry has operated successfully for years.
Completely agree the natural instinct is to rubber stamp. But in the language industry, their boss looks at metrics like “percent of translations edited” and translators reviewing machine translations will get flagged and loose the work if they are bypassing the expectations and not doing the job.
In other words this is a mostly solved problem. Put the responsibility on the worker for the AI’s output, and the worker will care as much about its output as they care about their job. Which also applies in the software field. Employers are generally fine with Copilot, etc, but it’s not an excuse for sh*t code. That same model can be applied in different contexts.
https://dftdigital.blog.gov.uk/2018/04/09/the-write-stuff-ho...
AI reads the letter, see if goes to the team dealing with X, Y, or Z, then it gets summarised and sent ready for answering.
I think in many ways this is the real story of AI: we have convinced the decision-makers of the world of the power of computing, but they don't know anything about computers, so they are wildly enthusiastic about a technology they understand - a program that makes a computer behave a little like a person.
(This isn't a complaint against AI.)