Large language models can do jaw-dropping things. But nobody knows why
technologyreview.com
technologyreview.com
Not so. Only if the same data is fed to the model.
There first example, arithmetic, we know actually benefits from over fitting, but even then it isn't great.
We now know that over Parameterization and double decent is less lossy compression.
We know that feed forward networks are TC^0 and we know that CoT etc... are limited to polynomial reductions.
The fact that we can't predict exactly what is learned in a specific instance is purely a function of computational complexity theory.
Attention we know is equivalent to exhaustive search.
Yes interpretability is hard and math explains that.
We know that the curse of dimensionality means LLMs are using extrapolation as an example.
https://arxiv.org/abs/2110.09485
This isn't meant to diminish the very impressive advances that have been made.
But it is not a complete god box, we can understand the underlying properties that lead to models being found.
But IMHO pretending that it is something that is magical seriously diminishes the hard work that was required to get where we are today.
Honestly, I wonder why no corporation has hit on intentionally hiring literally crazy people to work for them, but paying them less than sane people. It seems like it would be an innovative way to drive cost savings and efficiency.
I am aware you are doing the "modest proposal" bit. Still: fuck off. People like you are why I never ask for reasonable accommodations for my disability, even when I am having serious health problems.
That said, there is some truth to what the GP posted. The recycling center in my area works with a local adult services agency to place people with severe autism and other mental health conditions with a similar level of impact on their lives into positions where they can productive. Often that's sorting inbound recyclables. They pay them a bit below minimum wage. That seems deplorable on its face, but after some consideration I think it's a net benefit to all involved.
The company isn't getting the same productive output that they would from a typical employee, and they wouldn't break even if they paid them minimum wage.
The people doing the work are legitimately getting an opportunity they wouldn't otherwise have - it's difficult to find productive work for many of them, especially those that are non-verbal. They're able to earn a living. For most of them, it's the first time in their lives they've been able to say they have a job. The people I've spoken to in that position were all obviously proud of their work, and their pay is supplemented by welfare programs. They took and held the job because they wanted to be useful or simply wanted some spending money for themselves. The alternative for them was to be fully reliant on public assistance programs and sit idle all the time.
At a higher level, I've heard of companies with recruiting programs explicitly designed to seek out high-functioning people with autism, blindness, and deafness. I'm told those were based on positive experiences with past employees with those disabilities, and they had open positions to fill that they knew from that experience those people could perform the roles well.
While my current employer has no specific program for it, I believe we have a disproportionate number of employees with ADHD, depression, and other mental health condition that tend to do well with minimal accommodation. That's why I'm here - I really enjoy working somewhere that "I can't focus at all today, I'm logging out for the day" and "please reschedule this meeting; I'm hyperfocused on another task and don't want to break that" are equally valid statements that are universally understood and accepted.
Fair warning: when people like you break down and go AWOL, you end up on the radar of people like me. Then it gets handled as a semi-public security incident instead of the confidential medical issue it should have been. Once I reach out to your colleagues trying to figure out what the fuck is going on, rumors start spreading.
Your aversion to accepting accommodations only hurts you. If you need help, you must ask. Losing control of your own narrative is nobody's fault but yours.