ChatGPT-Authored Legal Filing “Replete with Citations to Non-Existent Cases"
reason.com
reason.com
A lawyer just straight making up citations of case law, even if hallucinated by an AI assist tool, should have similar repercussions re: license to practice law.
There needs to be no leniency for people misusing 'well, the AI told me...' as a crutch for actual knowledge and expertise in a professional field.
I think that should apply to any tool, but sadly it seems even most developers aren't immune to blindly trusting their tools, judging by what I've seen over the years.
As a memorable warning sign says: "The machine has no brain. Use your own."
First lesson of the first day of the first year in my CS degree -- the professor starts saying something to the effect of: "I want you to think of computers as meaningless machines doing meaningless operations on meaningless symbols".
To this day, that sentence is a massive influence in how I think about computers and technology in general.
Every time I open the thing I get a popup with "While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice."
If it gives you what you want, or something that looks like what you want or need, it gets harder and harder to say no and not use it.
A few times I was tempted to just go it blindly and I’m glad I didn’t.
TL;DR: verifying LLM output gets tiring.
If we use the same principals but fine tune the (or create a new) model with the appropriate legal information, I think we’ll see far better results. If not now, then in the very near future. Companies like LexisNexis[1] have a strong incentive to make this happen.
In short, for the UK at least, no, nobody needs qualifications to do that kind of thing. Though it is clearly a good idea.
Lawyer sues openAI for megabucks because they didn’t have a warning about using ChatGPT as a paralegal and they got sanctioned by the court.
Thirty years of practicing law and they (plural, two lawyers involved) “accidentally” submitted the output from a multi-billion dollar company’s toy without doing any checks?
> ChatGPT may produce inaccurate information about people, places, or facts.
That really should be sufficient as "not fit for legal work without full review".
Look how many people thought Tesla had full self driving when it says in the user agreement that they don't.
" Full Self-Driving Capability
Build upon Enhanced Autopilot and order Full Self-Driving Capability on your Tesla. This doubles the number of active cameras from four to eight, enabling full self-driving in almost all circumstances, at what we believe will be a probability of safety at least twice as good as the average human driver. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat. For Superchargers that have automatic charge connection enabled, you will not even need to plug in your vehicle.
All you will need to do is get in and tell your car where to go. If you don’t say anything, the car will look at your calendar and take you there as the assumed destination or just home if nothing is on the calendar. Your Tesla will figure out the optimal route, navigate urban streets (even without lane markings), manage complex intersections with traffic lights, stop signs and roundabouts, and handle densely packed freeways with cars moving at high speed. When you arrive at your destination, simply step out at the entrance and your car will enter park seek mode, automatically search for a spot and park itself. A tap on your phone summons it back to you.
Please note that Self-Driving functionality is dependent upon extensive software validation and regulatory approval, which may vary widely by jurisdiction. It is not possible to know exactly when each element of the functionality described above will be available, as this is highly dependent on local regulatory approval. Please note also that using a self-driving Tesla for car sharing and ride hailing for friends and family is fine, but doing so for revenue purposes will only be permissible on the Tesla Network, details of which will be released next year. "
https://web.archive.org/web/20171116234123/https://www.tesla...
They (hopefully) will simply make their job easier but there are hidden risks: identifying well written "hallucinations" is not trivial.
Tip for the AI brigade: Assign a confidence level to segments of generated text.
you say this as if it’s a trivial feature that AI developers have just not bothered to leave in
The important aspect is to be able to guide the "reviewer" where to focus on. It could still be a flawed suggestion but its better than nothing.
tip for the crypto brigade: make your coins solid investments
Text generators are not reasoning machines. To assign a confidence level, you need a tree of facts and deductions. This is what you can get from an expert system.
With an expert system, you can in principle ask the system to explain its reasoning, and assign confidence to each step. ML systems are black boxes.
Expert systems are hard to train; you need a human expert, and probably a specialist in the expert system sitting beside him. And it takes a lot of work - so it's expensive.
I can imagine a hybrid expert system/machine-learning system, where the ML system is used to distil the expertise that is then used by the expert system to construct decision trees that are not opaque.
I wish a fraction of the money that gets thrown at LLM "toys" were instead invested in expert systems. I think expert systems research became unfashionable around the end of the 1980s, which is a crying shame.
But my suggestion is actually less ambitious: not to create confidence with reference to outside knowledge but purely on the basis of the training sample and its intrinsic statistics.
2.13 Overreliance
...despite GPT-4’s capabilities, it maintains a tendency to make up facts, to double-down on incorrect information, and to perform tasks incorrectly. Further, it often exhibits these tendencies in ways that are more convincing and believable than earlier GPT models (e.g., due to authoritative tone or to being presented in the context of highly detailed information that is accurate), increasing the risk of overreliance.
Overreliance occurs when users excessively trust and depend on the model, potentially leading to unnoticed mistakes and inadequate oversight. This can happen in various ways: users may not be vigilant for errors due to trust in the model; they may fail to provide appropriate oversight based on the use case and context; or they may utilize the model in domains where they lack expertise, making it difficult to identify mistakes. As users become more comfortable with the system, dependency on the model may hinder the development of new skills or even lead to the loss of important skills. Overreliance is a failure mode that likely increases with model capability and reach. As mistakes become harder for the average human user to detect and general trust in the model grows, users are less likely to challenge or verify the model’s responses.
Source: https://arxiv.org/abs/2303.08774
My annotated version: https://lifearchitect.ai/report-card/
Uncanny and compelling as it is conversationally, it really isn't that good.
[0] https://platform.openai.com/docs/guides/embeddings/what-are-...
Sounds like a bunch of effort. :/
ChatGPT is fun, and for some of what I do it's usable as-is, but for most people it'll be a tech demo for something that will be built in and hidden behind layers of stuff to make it suitable for their specific use-case. At least until the base models get a lot more capable.
half the complaints people have about chatGPT are because they’ve never bothered to try the—much more powerful—API
Something along the lines of "why bother looking at their API, when the existing thing doesn't work properly?".
You're saying it's better via the API, but the standard (non-api) approach would have to be good enough first to draw people in more I guess.
I think part of the problem is people are used to equate eloquence with intelligence and knowledge, and when that assumption doesn't hold, they assume it's all smoke and mirrors rather than explore its limits.
At the end of the day, it remains a sequence completion engine. An LLM can only care about the statistical likelyhood of sequences, that the entire universe it exists in.
And if a sequence making a wrong statement happens to be more likely than, or close enough for the set "heat" of the algorithm to, a sequence denoting a correct statement, then there is a chance the model will hallucinate.
If it were otherwise, we would already have a good solution for prompt injection attacks, and we don't.
When you compress vast amounts of text into a model and query the model you can only get back information that is already in the training data set. It's inherently backwards looking, it cannot produce lowered entropy forward in time. You can't get more knowledge out of it, only less then you inputted in the first place.
I have a colleague who is gungho chat bot coding. The guy is smart, but ever since he started using it, he's adopted some weird ideas about what code should look like which are based on easily googleable chatgpt lies and he refuses to hear anyone else out. It's kinda sad how confidently wrong chatbots can infect people and stunt their growth...
"We can't fly into space, the propellers will have nothing to push against"
"We have rockets for that, they're more expensive but don't have that limitation."
"I see we're reaching the 'you're holding it wrong' stage with propulsion."
To people who understand the technology your reply is fair. But it doesn't account for the millions of people who don't and are spewing manic sentiment about it's capabilities on the regular.
A lot of companies adopt this strategy for broken technology because it fosters a cult like group of zealots for a short period of time while they go off and try to fix everything. In my experience it usually doesn't last long.
EDIT: found another way in which you're illustrating my point - yeah propellers and LLMs are great for flying under a hard ceiling. We need something entirely different to reach orbit. You can't just tell people to bolt on rockets to propeller driven aircraft, you need to invent something entirely different, like spacecraft.
More seriously though, it works for general “filling” like “thank the team for reaching milestone”, but you really need to insert your real examples (which the lawyer didn’t). Wouldn’t trust it to make a full argument, like in this case.
IMHO, it doesn't work for that, because the manager will come across as insincere for not even saying a few honest words of thanks.
That doesn't prevent installing a poor leader, but eventually that particular one can be corrected, through coaching or freak elevator accident.
And maybe someday the business schools will teach people that being genuinely trustworthy, in all org chart directions, is a best practice in modern, effective organizations. More of the right people for that will be drawn to leadership. And more of the people who are predisposed to make organizations dysfunctional and miserable will either begin the process of improving, or be herded to roles in which they can do less damage.
The lawyer should lose his license for six months. But OpenAI should also face penalties for creating an Attractive Nuisance.