Worrying that this will no longer be the case.
GPT-4 can do that task for fractions of a penny per email now. It doesn't have to be perfect if its competing with nothing. I expect we'll see similar shops for any other high cost paper/trail business.
What I'd really love to implement is a way for GPT-4 to answer questions based on a corpus of "all our Confluence pages plus random other sources of documentation." Like with the legal document issue, it's a bit of a nonstarter right now given the proprietary nature of corporate documentation.
"Hey, has anyone worked on Problem X, and what was the outcome of their project"
https://old.reddit.com/r/ChatGPT/comments/12fiwaf/chat_gpt_w...
GPT-like systems will close that gap, and then comes all of the problems of automated law enforcement - Extrapolation from incomplete data, false positives from coindicences, interpretation errors, all that annoying stuff
No, it hasn't lol.
Take drones for example. The government got really good at those because they made them jet-powered (lol) and blew a bunch of money on server-grade FPGA’s in each one of them.
You can’t really just buy a lot of GPUs to make an LLM work, you need iterative development of architecture and training methods.
Like maybe the government invented self-attention before 2017, but if they didn’t, then the constraint is training time, and the government has the same number of seconds as the rest of us.
Everything you use is from the military.
The government is good at lying, and making themselves ‘appear’ incompetent.
They secretly probably have a much further advanced quantum computer. Your viewpoint is limited to mainstream technology and mainstream science.
The military invented the nuclear bomb yes. But Fermi did most of his thinking work in Italy before the Manhattan project. He got money thrown at him once he got here.
As for semiconductor devices, it was Bell Labs and TI.
Coding is an ambiguous concept that wasn’t really invented, but if it were, it would have first appeared in programmable looms.
The military likes to take credit for things, but really all they do is throw money at existing inventions.
I’m sure they’re throwing a bunch of money at Transformers now, but who are all these uncredited super geniuses who invent things and then let randos at Google take the credit/earn the money?
They made some mistakes in the 40's and 50's to where they had nuclear secrets stolen by the Russians. And ever since has been hyper compartmentalized.
It would not surprise me if in the 90's or 00's they had an internal working LLM, considering all the puzzle pieces. You will never hear about classified tech unless it's a bomb, gets leaked. (See code breaking machines declassed after 70+ years)
In a hypothetical scenario, a military organization might want to conceal its use of a large language model (LLM) for intelligence gathering and analysis.
Another scenario is the military's current interest in everything quantum. Quantum computers for example (you wouldn't want another nation being first and pirate baying out our secrets, would you) so there is an extreme national security importance of being first.
And to be first, you need to have the smart people, which the military has. There is a reason China struggles with jet engines 80 years after their invention, and still can't make nuclear carriers. While the US navy works on things like this: https://www.navair.navy.mil/foia/sites/g/files/jejdrs566/fil...
Jet engines can be solved with money.
The actual steps of making an LLM require complex math and you can’t just pay people to make better math.
And if you could, wouldn’t those people decamp for industry and become literal trillionares?
So your implication is that the military is full of unnamed linear algebra, systems engineering, and linguistics super geniuses and these people never leave, never talk about their work publicly, never publish anything ever, and they're all cool with their huge innovations being kept away from the public forever? All because military IP regulation?
And none of these effective state prisoners ever defect to China (where they could live like royalty) because...
If that is your angle, I very much agree it's possible some deep black project exists that (once) looked into this. In the UFO lore, there are many stories about these advances (tr3b etc).
But all of it is orthogonal to LLMs though. Picking one exceptional area that the military is great at (aerospace) does not suddenly make the military exceptional in other areas like AI.
https://www.youtube.com/watch?v=-wU7nPDcTuY&t
See above video, even code breaking machines are kept a dark secret. LLM's that could make analytical decisions about war and strategy, must have started with the military first. It explains its massive data gathering operations in the 2000's. And it explains why some countries separation to make their own internet, away from what really is the US-Owned World Wide Web.
U.S. military has engaged in the commercialization of top-secret technologies (after it's considered obsolete by military standards), often by collaborating with private companies or research institutions.
Even the Manhattan project had nuclear research going on in public universities at the time.
Nothing of the sort here for the attention mechanism which underpins LLMs we know today.
Fundamental research isn't something you just throw money at and acquire. All we had back then were cleverbot and other expert systems.
The military is responsible for most the technology we use and talk about today. The government may appear incompetent, but we’re living off military hand-me-downs, the entire world is
If today's hardware was available 20 yrs ago, this would've been possible just like the moon landing could've been faked if it took place 20+ yrs later. The technology wasn't available at the time (GPUs in this case, and generally no experience in doing such advanced trick techniques for movies back then)
These models are having such a strong effect now because we've finally got the hardware to run them
They are not deep learning/neural nets.
Also fun fact as a pedant tax: Symantec is so named because they started out as transcription software, hit a wall, and pivoted to security SW.
[1] https://www.theguardian.com/technology/2011/mar/17/us-spy-op...
[2] https://boingboing.net/2015/06/22/gchqs-psy-ops-squad-target...
Because the hardware has not existed.
This said by accident I've seen hardware that was brought to a testing company by federal marshals that was massively parallel custom hardware that was likely for signal processing a lot of channels at once. So there is plenty of custom hardware out there, but these items have not been produced at the scale needed (from what anyone can tell) and, again from what we can tell, they don't have the general processing capability that GPU/TPU driven LLMs have.
not exactly. Where do you think all those budget trillions that don't have to be accounted for goes into? the FBI+NSA (=CIA but for citzens) have infinite resources.
All the overhead they have is to make sure a small subset of the citizens are not impacted. Snowden goes into this in some detail when talking about day to day operations. The norm is to extend the net as wide as possible, until you reach some politician or government agency.
They've created a huge library of unorganized data. The difference here is they now can spawn a million untiring AI private investigators / librarians to organize this information into coherent "case files".
At least for me, until this point I've had a feeling of anonymity in the idea that, while my data is being slurped up, I'm just one data point in a sea of other 'normal' people. There would be little value in spending government time and effort tying all of the web detritus together for me. The juice would definitely not be worth the squeeze.
However, when the cost of this effort is nearly zero, that now becomes a different story. The balance of power between government and the people it rules is going to radically shift.
https://www.cnn.com/2021/04/29/tech/nijeer-parks-facial-reco...
Shotspotter has been billed as a “system of sensors, software, AI and expert human review that accurately detects, locates and alerts police to gunfire”, and the company behind it (formerly “Shotspotter” was the company name, its recently been renamed “Soundthinking”) has a number of other AI-involved law enforcement products now, as well.
That's already how it worked on platforms like mturk and uhrs, lots of the work was transcribing audio dumps from microphones built into computers/phones/smart home devices. UHRS especially had a lot of that (it's owned by MS) as well as search engine grading type work. They also certainly do not pay well, I'd imagine that in practice there isn't much cost difference to paying a bunch of bored people to do it vs the compute cost for running an AI model to do it, but the AI model will be vastly more accurate and will work 24/7.
Perhaps history will show that the NSA made algorithmic breakthroughs a few years ago and realized what was coming, so political policy was crafted to stymie Chinese progress in this field, and what we're seeing in the public sphere from companies like openAI is a managed release of the technology into the public, openAI at least managing to independently discover the same breakthroughs that the NSA made a a few years ago.
Most people don't really think about things that don't affect their day-to-day lives. This includes the specifics of how Governments might run a mass surveillance plan.
Imagine you've sent an email about transporting a friend's daughter across state lines to get a medically-necessary abortion. Or if you prefer, imagine you've arranged via email to "lose" some firearms which don't comply with your state's new assault weapons ban.
Pre-LLMs, trying to find these sorts of emails was very hard. A simple text search for "abortion" or "gun" is going to come up with far more emails where two family members got into a political debate, than emails about lawbreaking. Big Brother will find a few such emails here and there by chance, but the vast majority of such incriminating emails will simply be lost in the pile.
Enter LLMs, and Big Brother can feed some of the incriminating emails found my chance into a training dataset along with a bunch of non-incriminating emails, and teach the AI to find incriminating emails, and then apply the model to the entire list of emails and get a nicely filtered list of only the emails which are incriminating, further tuning the model by adding emails it gets wrong to the training dataset when they are found.