TSMC execs allegedly dismissed OpenAI CEO Sam Altman as 'podcasting bro'
tomshardware.com
tomshardware.com
Both of the first two comments you posted seemed to be positive about the NYT article over the TH one — I had not previously encountered “insightful” used negatively, or “losing nuance” used positively — so perhaps a different choice of words would have made it clearer (to me).
Behind OpenAI's plan to make A.I. flow like electricity - https://news.ycombinator.com/item?id=41663562
Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html
If a machine spits out some plausible looking text (or some cookie-cutter code copy-pasted from Stack Overflow) the human brain is basically hardwired to go "wow this is a human friend!". The current LLM trend seems designed to capitalize on this tendency towards sympathizing.
This is the same thing that made chatbots seem amazing 30 years ago. There's a minimum amount of "humanness" you have to put in the text and then the recipient fills in the blanks.
This is not a reasonable take on the current capabilities of LLMs.
Then there's chain-of-thought being positioned as the next big step forwards, which works by throwing more inferencing at the problem, so that cost can't be amortized over time like training can...
2) Absolutely. Thats like one hour of an engineer salary for a whole month.
Isn't each new model bigger and heavier and thus requries more compute to train?
Until the competition outcompetes you with their new model and you have to train a new superior one, because you have no moat. Which happens what, around every month or two?
> the hardware is getting much better and prices will fall drastically once there is a bit of a saturation of the market or another company starts putting out hardware that can compete with NVIDIA
Where is the hardware that can compete with NVIDIA going to come from? And if they don't have competition, which they don't, why would they bring down prices?
Huge margins lead to a lot of competition trying to catch up, which is what makes market economies so successful.
Eventually one of you runs out of money, but your customers keep getting better models until then; and if the loser in this race releases the weights on a suitable gratis license then your businesses can both lose.
But that still leaves your customers with access to a model that's much cheaper to run than it was to create.
Google however does not sell these, you can only lease time on them via GCP.
Raising the max intelligence of the models tends to raise the intelligence of all the models via distillation.
Short term pricing inefficiency is not relevant to long term impact.
If there is a common shift to using additional runtime compute to improve quality of output, such as OpenAI's GPT-o1, then FLOPs required goes up massively (OpenAI has said it takes exponential increase in FLOPS/cost to generate linear gains in quality).
So, while costs will of course decrease, those $20-30K NVIDEA chips are going to be kept burring, and are not going to pay for themselves ...
This may end up like the shift to cloud computing that sounds good in theory (save the cost of running your own data center), but where corporate America balks when the bill comes in. It may well be that the endgame for corporate AI is to run free tools from the likes of Meta (or open source) in their own datacenter, or maybe even locally on "AI PCs".
Drop the S, I think. There’s no time dimension.
And FLOP is a generalized capability meaning you can do any operation. Hardware optimizations for ML can deliver the same 100B computations faster and cheaper by not being completely generalized. Same way ray tracing acceleration works: it does not use the same amount of compute as ray tracing in general CPU’s.
Still, even with modern accelerators it's lot of computation, and is what drives the price per token of larger models vs smaller ones.
Llama, Mixtral, Stable diffusion and Flux are a lot of fun and free to run locally, you should try them out.
Let me use CG rendering as an example. Back in the day only the big companies could afford to do photoreal 3D rendering because only they had access to the compute and even then it would take days to render a frame.
Eventually people could do these renders at home with consumer hardware but it still took forever to render.
Now we can render photoreal with path tracing at near realtime speeds.
If you could go back twenty years and show CG artists the Unreal Engine 5 and show them it’s all realtime they would lose their minds.
I see the same for A.I., now it’s only the big companies that can do it, then we will be able to do it at home but it will be slow and finally we will be able to train it at home for quick and cheap.
It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else.
Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum).
His answer was "a few days".
What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.
Step outside of building basic web/CRUD apps and its accuracy drops off substantially.
Also almost every library it uses is old and insecure.
I have also been developing for 20+ years.
And have heard the exact same thing about IDEs, Search Engines, Stack Overflow, Github etc.
But in my experience at least how fast I code has never been the limiting factor in my project's success. So LLMs are nice and all but isn't going to change the industry all that much.
If nothing really changes in 3-5 years, then I'd call it a flop. But the writing is on the wall that "scale = smarts", and what we have today still looks like a foundational stage for LLM's.
> If nothing really changes in 3-5 years, then I'd call it a flop
Transformers have been used for what 6 years now? Will you in 6 years say "I'll decide if they don't change the world in another 6 years?"
Working with AI-generated code to add new features feels like working with Dreamweaver-generated code, which was also unpleasant. It's not written the same way a human would write it, isn't written with ease of modification in mind, etc.
I've tried using LLMs for some libraries I'm working on, and they failed miserably. Trying to make an LLM implement a trait with a generic type in Rust is a game of luck with very poor chances.
I'm sure LLMs can massively speed up tasks like front-end JavaScript development, simple Python scripts, or writing SQL queries (which have been written a million times before).
But for anything even mildly complex, LLMs are still not suited.
Succeeding on the most common tasks (which isn't exactly what you said) is identical to "they're useful".
But it's also utterly failed to handle mundane tasks, like porting legacy code from one language and ecosystem to another, which is frankly surprising to me because I'd have assumed it would be perfectly suited for that task.
But the worst LLMs? One of my personal tests is "write Tetris as a web app", and the worst local LLM I've tried, started bad and then half way through switched to "write a toy ML project in python".
It’s a very useful tool, not magic.
front-end JS can easily also become very complex
I think a better metric is how close you are to reinventing a wheel for the thousands time. Because that is what LLMs are good at: Helping you write code which nearly the same way has already been written thousands of times.
But that is also something you find in backend code, too.
But that is also something where we as a industry kinda failed to produce good tooling. And worse if you are in the industry it's kinda hard to spot without very carefully taking a hounded (mental) steps back from what you are used to and what biases you might have.
We should not have an entire industry of 10,000,000 devs reinventing the JS/React/Spring/FastCGi wheel. Im sure those humans can contribute in much better ways to society and progress.
I'd have said the opposite. I think LLMs facilitate disposable code. It might use the same paradigms and patterns, but my bet is that most LLM written code is written specifically for the app under development. Are there LLM written libraries that are eating the world?
The code itself is not clean and reusable across implementations, but you don't even need that clean packaged library. You just have an LLM regenerate the same code for every project you need it in.
The LLM itself, combined with your prompts, is effectively the reusable code.
Now, this generates a lot of slop, so we also need better AI tools to help humans interpret the code, and better tools to autotest the code to make sure it's working.
I've definitely replaced instances where I'd reach for a utility library, instead just generating the code with AI.
I think we also have an opportunity to merge the old and the new. We can have AI that can find and integrate existing packages, or it could generate code, and after it's tested enough, help extract and package it up as a battle tested library.
E.g. let's say I'm working on a production thing and features/bugfixes accumulate and some file in the codebase starts to resemble spaghetti. The LLM can help me unfuck that way faster and get to a state of very clean code, across many files at once.
Could potentially mean just a change in time allocation/priority. As it's easier and faster to locate and potentially resolve issues later, it is less important for code to be consistent and perfectly documented.
Not fool proof and who knows how that could evolve, but just an alternative view. One of these big names in the industry said we'll have AGI when it speaks it's own language. :P.
LLMs are definitely suited for tasks of varying complexity, but like any tool, their effectiveness depends on knowing when and how to use them.
SQL is a good target language because the translation from ideas (or written description) is more or less linear, the SQL engine uses entirely different techniques to turn that query into a set of relational operators which can be rewritten for efficiency and compiled or interpreted. The LLM and the SQL engine make a good team.
1. Aasked ChatGPT to write a simple echo server in C but with this twist: use io_uring rather than the classic sendmsg/recvmsg. The code it spat out wouldn't compile, let alone work. It was wrong on many points. It was clearly pieces of who-knows-what cut and pasted together. However after having banged my head on the docs for a while I could clearly determine from which sources the code io_uring code segments were coming. The code barely made any sense and it was completely incorrect both syntactically and semantically.
2. Asked another LLM to write an AWS IAM policy according to some specifications. It hallucinated and used predicates that do not exist at all. I mean, I could have done it myself if I just could have made predicates up.
> But for anything even mildly complex, LLMs are still not suited.
Agreed, and I'm not sure we are any close to them being.
This is probably why there’s such a divide when you try to talk about software dev online. One camp believes that it boils down to duct taping as many ready made components together all in pursuit of impact and business value. Another wants to really understand all the moving parts to ensure it doesn’t fall apart.
Personally, I just use the hell out of Django for that. And since tools like that are already ridiculously productive, I don't see much upside from coding assistants. But by and large, so many of our tools are so surprisingly _bad_ at this, that I expect the LLM hype to have a lasting impact here. Even _if_ the solutions aren't actually LLMs, but just better tools, since we reconfigured how long something _should_ take.
Bad tools often falls in three categories. Too simple, too complex, or unsuitable. For the last two, you'd better switch but there's the human element of sunken costs.
> Current LLMs fail if what you're coding is not the most common of tasks. And a simple web app is about as basic as it gets.
These two complexity estimates don’t seem to line up.
I've used LLMs to generate quite a lot of Rust code. It can definitely run into issues sometimes. But it's not really about complexity determining whether it will succeed or not. It's the stability of features or lack thereof and the number of examples in the training dataset.
What I meant by complexity is not "a task that's difficult for a human to solve" but rather "a task for which the output can't be 90% copied from the training data".
Since frontend development, small scripts and SQL queries tend to be very repetitive, LLMs are useful in these environments.
As other comments in this thread suggested: If you're reinventing the wheel (but this time the wheel is yellow instead of blue), the LLM can help you get there much faster.
But if you're working with something which hasn't been done many times before, LLMs start struggling. A lot.
This doesn't mean LLMs aren't useful. (And I never suggested that.) The most common tasks are, per definition, the most common tasks. Therefore LLMs can help in many areas, and are helpful to a lot of people.
But LLMs are very specialized in that regard, and once you work on a task that doesn't fit this specialization, their usefulness drops, down to being useless.
And for me that is the best case scenario, it takes away the part we have to code / solve already solved problems again and again so we can focus more on the other parts of software engineering beyond writing code.
Personally I much prefer Chatgpt. I give it specific small problems to resolve and some context. At most 100 lines of code. If it gets more the quality goes to shit. In fact copilot feels like chatgpt that was given too much context.
All of my experiences with LLMs have been that for anything that isn't a braindead-simple for loop is just unworkable garbage that takes more effort to fix than if you just wrote it from scratch to begin with. And then you're immediately met with "You're using it wrong!", "You're using the wrong model!", "You're prompting it wrong!" and my favorite, "Well, it boosts my productivity a ton!".
I sat down with the "AI Guru" as he calls himself at work to see how he works with it and... He doesn't. He'll ask it something, write an insanely comprehensive prompt, and it spits out... Generic trash that looks the same as the output I ask of it when I provide it 2 sentences total, and it doesn't even work properly. But he still stands by it, even though I'm actively watching him just dump everything he just wrote up for the AI and start implementing things himself. I don't know what to call this phenomenon, but it's shocking to me.
Even something that should be in its wheelhouse like producing simple test cases, it often just isn't able to do it to a satisfactory level. I've tried every one of these shitty things available in the market because my employer pays for it (I would never in my life spend money on this crap), and it just never works. I feel like I'm going crazy reading all the hype, but I'm slowly starting to suspect that most of it is just covert shilling by vested persons.
In theory I'm learning from the LLM during this process (much like a real code review). In practice, it's very rare that it teaches me something, it's just more careful than I am. I don't think I'm ever going to be less slap-dash, unfortunately, so it's a useful adjunct for me.
I use it to write test systems for physical products. We used to contract the work out or just pay someone to manually do the tests. So far it has worked exceptionally well for this.
I think the core issue of the "do LLMs actually suck" is people place different (and often moving) goalposts for whether or not it sucks.
It works well as a smart documentation search where you can ask follow-up questions or when you know what the output should look like if you see it but can't type it directly from the memory.
For code assistants (aka copilot / cursor), it works if you don't care about the code at all and ok with any solution if it's barely working (I'm ok with such code for my emacs configuration).
When they say treat it like an intern, I'm so confused. An intern is there to grow and hopefully replace you as you get promoted or leave. The tasks you assign to him are purposely kept simple for him to learn the craft. The monotonous ones should be done by the computer.
[0]: https://gist.github.com/simonw/97e29b86540fcc627da4984daf5b7...
After 20 years of being held accountable for the quality of my code in production, I cannot help but feel a bit gaslit that decision-makers are so elated with these tools despite their flaws that they threaten to take away jobs.
Lots of people are terrible at going from 0 to 1 in any project. Me included. LLMs helped me a lot solving this issue. It is so much easier to iterate over something.
And that’s fine if the dev realizes what’s going on but when they attribute their own quirks to AI magic, that’s a problem.
It's almost as if the horde of former kleptocurrency bros have found a promising new seam of fool's gold to mine
I did it in little pieces and started over with fresh context each time the LLM started to get off in the weeds. I'm very happy with the result. The code is clean and well commented, the tests are comprehensive and the app looks nice and performs well.
I could have done all this manually too but it would have taken longer and I probably would have skimped out on some tests and gave up and hacked a few things in out of expedience.
Did the LLM get things wrong on occasion? Yes. Make up api methods that don't exist? Yes. Skip over obvious standard straightforward and simple solutions in favor of some rat's nest convoluted way to achieve the same goal? Yes.
But that is why I'm here. It's a different style of programming (and one that I don't enjoy nearly as much as pounding the keyboard). It's more high level thinking and code review involved and less worrying about implementation detail.
It might not work as well in domains which training data doesn't exist in. Also certainly if someone expects to come in with no knowledge and just paste code without understanding, reading and pushing back, they will have a non working mess pretty shortly. But overall these tools dramatically increase productivity in some domains is my opinion.
If you aren’t on board then it looks impressive but flawed and not even close to living up to the hype.
I've been impressed with the ability to generate "throw away" code for testing out an idea or rapidly prototyping something.
I have a good mental map of the projects I work on because I wrote them myself. When new business problems emerge, I can picture how to solve them using the different components of those applications. If I hadn't actually written the application myself, that expertise would not exist.
Your colleague may have a working application, but I seriously doubt he understands it in the way that is usually needed for maintaining it long term. I am not trying to be pessimistic, but I _really_ worry about these tools crippling an entire generation of programmers.
That sounds quite useful. Does Cursor feed your entire project code (traversing all folders and files) into the context?
My constant suspicion is that most results people are so impressed with were just never validated.
> Analogously, I imagine future software dev to consist mostly of writing specs in natural language.
https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi...?
I like this take. I feel like a significant portion of building out a web app (to give an example) is boilerplate. One benefit of (e.g., younger) developers using AI to mock out web apps might be to figure out how to get past that boilerplate to something more concise and productive, which is not necessarily an easy thing to get right.
In other words, perhaps the new AI tools will facilitate an understanding of what can safely be generalized from 30 years of actual code.
I’d argue web frameworks don’t even help a lot in this regard still. They pile on more concepts to the leaky abstractions of the web. They’re written by people that love the web, and this is a problem because they’re reluctant to hide any of the details just in case you need to get to them.
Coworker argued that webdev fundamentally opposes abstraction, which I think is correct. It certainly explains the mountains of code involved.
It does seem inevitable that some large change will happen to our profession in the years to come. I find it challenging to predict exactly how things will play out.
Isn’t that the point? Degrade the user long enough that the competing user is on-par or below the competence of the tool so that you now have an indispensable product and justification of its cost and existence.
P.S. This is what I understood from a lot of AI saints in news who are too busy parroting productivity gains without citing other consequences, such as loss of understanding of the task or expertise to fact-check.
I'm not a mathematician, hell i did general maths at school. Currently I've been talking through scripting a method to mix dsd audio files natively without converting to tradional pcm. I'm about to use gpt to craft these scripts. There is no way I could have done this myself without years of learning. Now all I have to do is wait half a day so I can use my free gpt o credits to code it for me (I'm broke af so can't afford subs). The productivity gains are insane. I'd pay for this in a heartbeat if I could afford it.
In niche situations it's not helpful at all in writing code that works (or even close). It is helpful as a quick lookup for docs for libs or functions you don't use much, or for gotchas that you might otherwise search StackOverflow for answers to.
It's good for quick-and-dirty code that I need for one-off scripts, testing, and stuff like that which won't make it into production.
Yeah, if you want tic-tac-toe or snake, you can simply ask ChatGPT and it will spit out something reasonable.
But this is not much better than a search engine/framework to be honest.
Asking it to be "creative" or to tweak existing code however ...
The sad part is beginners using the boilerplate code won't get any practice building apps and will completely fail at the complex parts of an app OR try to use AI to build it and it will be terrible code.
10% more productive. What does that mean? If you mean lines of code, then it's an incredibly poor metric. They write more code, faster. Then what? What are the long-term consequences? Is it ultimately a wash, or even a detriment?
https://stackoverflow.blog/2024/03/22/is-ai-making-your-code...
Personally I am having a lot of fun, as an iOS developer, creating web games. No market in that, not really, but it's fun and I wouldn't have time to update my CSS and JS knowledge that was last up-to-date in 1998.
However, for a more typical software engineer, where every project is different, you have full lifecycle responsibility from design through coding, occasional production support, future enhancements, refactorings, updates for 3rd party library/OD updates, etc/etc, then how much of your time is actually spent pure coding (non-stop typing) ?! Probably closer to 10-25%, and certainly no-where near 100%. The potential overall time saving from a tool that saves, let's say, 10-25% of your code typing is going to be 1-5%, which is probably far less than gets wasted in meetings, chatting with your work buddies, or watching bullshit corporate training videos. IOW the savings is really just inconsequential noise.
In many companies the work load is cyclic from one major project to the next, with intense periods of development interspersed with quieter periods in-between. Your productivity here certainly isn't limited by how fast you can type.
If you pay Silicon Valley salaries this seems like a no-brainer. There are bigger time wasters elsewhere, but this is an easy win with minimal resistance or required culture change
Ok models already run locally; that aside, as the hosted ones are kinda similar quality to interns (though varying by field), the answer is "what you'd pay an intern". Could easily be £1500/month, depending on domain.
- ok it works, but it won't be useful.
- ok it's useful, but it won't scale.
- ok it scales, but it won't make any money.
- ok it makes money, but it's not going to last.
etc etc
After all, you could have used the exact same response in defense of web3 tech. That doesn't mean LLMs are fated to be like web3, but similarly the outcome that the current expenditure can be recouped is far from a certainty just because there are doubters.
I'm probably one of those "heavy users", though I've only been using it for a month to see how well it does. Here's my review:
Large completions (10-15 lines): It will generally spit out near-working code for any codemonkey-level framework-user frontend code, but for anything more it'll be at best amusing and a waste of time.
Small completions (complete current line): Usually nails it and saves me a few keystrokes.
The downside is that it competes for my attention/screen space against good old auto-completion, which costs me productivity every time it fucks up. Having to go back and fix identifiers in which it messed up the capitalization/had typos, where basic auto-complete wouldn't have failed is also annoying.
I'd pay about about $40 right now because at least it has some entertainment value, being technologically interesting.
If what I give it is too open ended, doesn't have enough info, etc, I'll still get a low quality output. Though I find I can steer it by asking it to ask clarifying questions. Asking it to build unit tests can help a lot too in bolstering, a few iterations getting the unit tests created and passing can really push the quality up.
100,000 ? 500,000 ?
If Copilot came for free and Azure cost a tiny bit more, nobody would even blink.
For ChatGPT in its current state, probably $1K/month.
When GPT4 was launched last year, the API cost was about $36/M blended tokens, but you can now get GPT4o tokens for about $4.4/M tokens, Gemini 1.5 Pro for $2.2/M or DeepSeek-V2 (as 21B A/236B W model that matches GPT4 on coding) for as low as $0.28/M tokens (over 100X cheaper for the same quality output over the course of about 1.5 years).
The just released Qwen2.5-Coder-7B-Instruct (Apache 2.0 licensed) also basically matches/beats GPT4 on coding benchmarks and quantized can not only can run at a decent speed on just about any consumer gaming GPU, but on most new CPUs/NPUs as well. This is about a 250X smaller model than GPT4.
There are now a huge array of open weight (and open source) models that are very capable and that can be run locally/on the edge.
> I'm more productive than ever before.
You realize that another way to read that sentence is "I am a really bad coder".
There are people who are getting “pretty rich” by trafficking humans, or selling drugs. Would you want to live in a society where such activities are encouraged? In the end, we need to look at technological progress (or any progress for that matter) as where it will bring us to in the future, rather than what it allows you to do now.
It also pisses me off that software engineering has such a bad reputation that everyone, from common folks to the CEO of nvidia, is shitting on it. You don’t hear phrases like “AI is going to change medicine/structural engineering”, because you would shit your pants if you had to sit in a dentist chair, while the dentist would ask ChatGPT how to perform a root canal; or if you had to live in a house designed by a structural engineer whose buddy was Claude. And yet, somehow, everyone is ready to throw software engineers under the bus and label them as "useless"/easily replaceable by AI.
But we're getting paid right? And the Prime van will keep delivering.
Op is addressing the hype that there is some linear path of improvement here and chatgpt 8.5 will be AGI.
To which people always seem to jump in with but it’s useful for me and makes me code faster. Which is fine and valid, just beside the point
I use Scheme a lot, but the 1970s MIT AI folks' contention that LISPs encapsulate the core of human symbolic reasoning is clearly ridiculous to 2020s readers: LISP is an excellent tool for symbolic manipulation and it has no intelligence whatsoever even compared to a jellyfish[1], since it cannot learn.
GPTs are a bit more complicated: they do learn, and transformer ANNs seem meaningfully more intelligent than jellyfish or C. elegans, which apparently lack "attention mechanisms" and, like word2vec, cannot form bidirectional associations. Yet Claude-3.5 and GPT-4o are still unable to form plans, have no notions of causality, cannot form consistent world models[2] and plainly don't understand what numbers actually mean, despite their (misleading) successes in symbolic mathematics. Mice and pigeons do have these cognitive abilities, and I don't think it's because God seeded their brains with millions of synthetic math problems.
It seems to me that transformer ANNs are, at any reasonable energy scale, much dumber than any bird or mammal, and maybe dumber than all vertebrates. There's a huge chunk we are missing. And I believe what fuels AI boom/bust cycles are claims that certain AI is almost as intelligent as a human and we just need a bit more compute and elbow grease to push us over the edge. If AI investors, researchers, and executives had a better grasp of reality - "LISP is as intelligent as a sponge", "GPT is as intelligent as a web-spinning spider, but dumber than a jumping spider" - then there would be no winter, just a realization that spring might take 100 years. Instead we see CS PhDs deluding themselves with Asimov fairy tales.
[1] Jellyfish don't have brains but their nerve nets are capable of Pavlovian conditioning - i.e., learning.
[2] I know about that Othello study. It is dishonest. Unlike those authors, when I say "world model" I mean "world."
I need to be convinced that an LLM is smarter than a honeybee before I am willing to even consider that it might be as smart as a human child. Honeybees are smart enough to understand what numbers are. Transformer LLMs are not. In general GPT and Claude are both dramatically dumber than honeybees when it comes to deep and mysterious cognitive abilities like planning and quantitative reasoning, even if they are better than honeybees at human subject knowledge and symbolic mathematics. It is sensible to evaluate Claude compared to other human knowledge tools, like an encyclopedia or Mathematica, based on the LLM benchmarks or "demonstrated LLM abilities." But those do not measure intelligence. To measure intelligence we need make the LLM as ignorant as possible so it relies on its own wits, like cognitive scientists do with bees and rats. (There is a general sickness in computer science where one poorly-reasoned thought experiment from Alan Turing somehow outweighs decades of real experiments from modern scientists.)
[1] People dishonestly claim LLMs fail at counting because of minor tokenization issues, but
a) they can count just fine if your prompt tells them how, so tokenization is obviously not a problem
b) they are even worse at counting if you ask them to count things in images, so I think tokenization is irrelevant!
But at the same time there is a lot of value to capture here by building solid applications around the capabilities that already exist. It might be a winter more like the "winter" image recognition went through before multimodal LLMs than the previous AI winter
a) childish motivated reasoning led people to think a fairly simple technology could solve profoundly difficult business problems in the real world
b) a culture of "number goes up, that's just science"
c) uncritical tech journalists who weren't even corrupt, just bedazzled
In particular I don't think generative AI is like cryptocurrency, which was always stupid in theory, and in practice it has become the rat's nest of gangsters and fraudsters which 2009-era theory predicted. After the dust settles people will still be using LLMs and art generators.
but if you remove implicit subventions from the AI/AGI hype then for many such tools the cost to benefit calculation of creating and operating will become ... questionable
furthermore the places where such tools tend to shine the most often places where the IT industry has somewhat failed, like unnecessary verbose and bothersome to use tools, missing tooling and troublesome code reuse (so you write the same code again and again). And this LLM based tools are not fixing the problem they just kinda hiding it. And that has me worried a bit because it makes it much much less likely for the problem to ever be fixed. Like I think there is a serious chance for this tooling causing the industry to be stuck on a quite sub-par plato for many many years.
So while they clearly help, especially if you have to reinvent the wheel for a thousands time, it's hard to look at them favorably.
E.g. I feel like it should be possible to first blast out a lot of repetitive code and then for LLM to go over all of it and abstract it reasonably, while tests are still passing.
How will that ever get solved, in this universe? Look at what C++ does to C, what TypeScript does to JavaScript, what every standard does to the one before. It builds on top, without fixing the bottom, paving over the holes.
If AI helps generate sane low level code, maybe it will help you make less buffer overflow mistakes. If AI can help test and design your firewall and network rules, maybe it will help you avoid exposing some holes in your CUPS service. Why not, if we're never getting rid of IP printing or C? Seems like part of the technological progress.
Similar this law isn't really a law for a good reason, it doesn't always work. Not everything gets cheaper (in a relevant amount) at scale.
No offense, but I have only seen people who barely coded before describe being "very productive" with AI. And, sure, if you dabble, these systems will spit out scripts and simpler code for you, making you feel empowered, but they are not anywhere near being helpful with a semi-complex codebase.
Let’s see in some years… long winter ahead.
- Generate boilerplate
- Generate extremely simple code patterns. You need a simple CRUD API? Yeah, it can do it.
- Generate solutions for established algorithms. Think of solutions for leetcode exercises.
So yeah, if that's your job as a developer, that was a massive productivity boost.
Playing with anything beyond that and I got varying degrees of failure. Some of which are productivity killers.
The worst is when I am trying to do something in a language/framework I am not familiar with, and AI generates plausibly sounding but horribly wrong bullshit. It sends me in some deadends that take me a while to figure out, and I would have been better just looking it up by myself.
- Generate boilerplate : Snippets, templates, and code generators
- Generate extremely simple code patterns : Frameworks
- Generate solutions for established algorithms : Libraries.
My point is that I don't think AI can meaningfully output code that would be useful beyond that, because that code is not available in its training data.
Whenever I see people going on about how AI made then super productive, the only thing I ask myself is "My brother in Christ, what the fuck are you even coding?"
Like it will generate code like `x && Array.isarray(x)` because `x && x is something` is a common pattern I guess - but it's completely pointless in this context.
It will often do roundabout shit solutions when there's trivial stuff built into the tool/library when you ask it to solve some problem. If you're not a domain expert or search for better solutions to check it you'll often end up with slop.
And the "reasoning" feels like the most generic answers while staying on topic, like "review this code" will focus on bullshit rather than prioritizing the logic errors or clearing up underlying assumptions, etc.
That said it's pretty good at bulk editing - like when I need to refactor crufty test cases it saves a bunch of typing.
It's like looking at the first version of an IDE that got intellisense/autocomplete and deciding that we'll be able to write entire programs by just pressing tab and enter 10,000 times.
I do not claim to know what the future holds, but I do feel the clock is ticking on the AI hype. OpenAI blew people's minds with GPTs, and people extrapolated that mind-blowing experience into a future with omniscient AI agents, but those are nowhere to be seen. If investors have AGI in mind, and it doesn't happen soon enough, I can see another winter.
Remember, the other AI winters were due to a disconnect between expectations and reality of the current tech. They also started with unbelievable optimism that ended when it became clear the expectations were not reality. The tech wasn't bad back then either, it just wasn't The General Solution people were hoping for.
They seem very good at writing SQL for example. All the commas are in the right place and exactly the right amount of brackets square curly and round. But when they get it wrong, it really shows up the lack of intelligence. I hope the froth and bubble in the marketing of these tools matures into something with a little less hyperbole because they really are great just not intelligent.
Coding AI assistants have done some impressive things, I’ve been amazed at how they sniffed out some repetitive tasks I was hacking on and I just tab completed pages of code that was pretty much correct. There is use. I pay for the feature. I don’t know if it’s worth 35% of the world’s energy consumption and all new fabrication resources over the next handful of years being dedicated to ‘ai chips.’ We arent looking for a better 2.0, we are expecting an exponentially better “2.0” and those are very rare.
I don't think the bulk of this VC money is predicated on AGI being around the corner.
But the general trend hopping nature of big VC money is real. Still, VCs manage to continue to make a profit despite this, otherwise the industry would have died off or shrunk the 10 other years HN critiqued this behaviour, so on the whole they must be doing something right.
There are so many things that can be automated out there that currently aren't. Other industries are extremely manual and process driven still. Many here tend to underestimate this.
Some programmers here will argue it's error prone or creating technical debt but most people don't care, if it works it works, one can worry about it breaking in 5 years time after its saved you considerably time and money.
Uhh, that's a pretty big change.
Who are you and what are you being so productive in?
These code assistants are wholly unable to help with the day to day work I do.
Sometimes I use them to remind me what flags to use with a tarball[0], so replaced SO, but anything of consequence or creativity and they flounder.
What are you getting out of this excess productivity? A pay raise? More time with your loved ones?
[0] https://xkcd.com/1168/ (addressing the tool tip, but hilariously, in regards the comics content that would be a circumstance where I would absolutely avoid trusting one of these ‘assistants’)
Outside of very very short isolated template creation for some kind of basic script or poorly translating code from one language to another, they have wasted more time for me than they saved.
The area they seem to help people, including me, the most in is giving me code for something I don't have any familiarity with that seems plausible. If it's an area I've never worked in before, it could maybe be useful. Hence why the less breadth of knowledge in programming you have, the more useful it is. The problem is that you don't understand the code it produces so you have to entirely be reliant on it, and that doesn't work long term.
LLMs are not and will not be ready to replace programmers within the next few years, I guarantee it. I would bet $10k on it.
Either you pay more and more to keep your job as it gets better, or the company pays any amount for it so they can replace you over and over as a barely useful cog.
The current state of it being cheap only exist as it is in beta and they need more info from you, the expert, until it no longer needs you
Our everyday lives should make it evident how much the working of our brain doesn't resemble that of our computers. Our experiences change our brains somehow but exactly how we don't have the faintest idea about and we can re-live these experiences somewhat which creates a memory but the mechanism is by no means perfect. There's the Mandela Effect https://pubmed.ncbi.nlm.nih.gov/36219739/ and of course "tip of my tongue" where we almost remember a word and then perhaps minutes or hours later it just bursts into our consciousness. If it's a computer why is learning so hard? Read something and bam, it's written in your memory, right? Right? Instead, there's something incredibly complex going on, in 2016 an fMRI study was made among the survivors of a plane crash and large swaths of the brain lit up upon recall. https://pubmed.ncbi.nlm.nih.gov/27158567/ Our current best guess is somehow its the connections among neurons which change and some of these connections together form a memory. There are 100 trillion connections in there so we certainly have our task cut.
And so we are here where people believe they can copy human intelligence when they do not even know what they are trying to copy falling for the latest metaphor of the workings of the human brain believing it to be more than a metaphor.
At the end of the day, does it matter? If humans can be fooled by artificial intelligence in pretty much all areas, and that intelligence surpasses ours by every possible measurement, does it really matter that it's not powered by biological brains? We haven't quite reached that stage yet, but I don't think this will matter when we do.
This is just preposterous. You can be fooled if you have no knowledge in the area but that's about it. With current tech there is, there can not be anything novel. Guernica was novel. No matter how you train any probabilistic model on every piece of art produced before Guernica it'll never ever create it.
There are novel novels (sorry for the pun) every few years. They delight us with genuinely new turns of prose, unexpected plot twists etc.
Also harken to https://garymarcus.substack.com/p/this-one-important-fact-ab... which also happens to include a verb made up on spot.
And yes we have cars which move faster than a human can but they don't compete in high jumps or climb rock walls. Despite we have a fairly good idea about the mechanical workings of the human body, muscles and joints and all that we can't make a "tin man", not by far. As impressive as Boston Dynamics demos are they are still very very far from this.
I wasn't talking about current tech, which is obviously not at human levels of intelligence yet. I would still say that our progress in the last 100 years, and the last 50 in particular, has been astonishing. What's preposterous is expecting that we can crack a problem we've been thinking about for millennia in just 100 years.
Do you honestly think that once we're able to build AI that _fully_ mimics humans by every measurement we have, that we'll care whether or not it's biological? That was my question, and "no" was my answer. Whether we can do this without understanding how biological intelligence works is another matter.
Also, AI doesn't even need to fully mimic our intelligence to be useful, as we've seen with the current tech. Dismissing it because of this is throwing the baby out with the bath water.
What made you think that is measurable and if it is then we can build something like that ever?
I already linked https://garymarcus.substack.com/p/this-one-important-fact-ab... did you read it?
What makes you think it isn't, and that we can't? The Turing test was proposed 75 years ago, and we have many cognitive tests today which current gen AI also passes. So we clearly have ways of measuring intelligence by whatever criteria we deem important. Even if those measurements are flawed, and we can agree that current AI systems don't truly understand anything but are just regurgitation machines, this doesn't matter for practical purposes. The appearance of intelligence can be as useful as actual intelligence in many situations. Humans know this well.
Yes, I read the article. There's nothing novel about saying that current ML tech is bad at outliers, and showcasing hallucinations. We can argue about whether the current approaches will lead to AGI or not, but that is beside the point I was making originally, which you keep ignoring.
Again, the point is: if we can build AI that mimics biological intelligence it won't matter that it's not biological. And a sidenote of: even if we're not 100% there, it can still be very useful.
Hydraulics, gear systems, and computers are all Turing complete. If you're not a dualist, you have to believe that each of these would be capable of building a brain.
The history described here is one where humans invent a superior information processor, notice that it and humans both process information, and conclude that they must be the same physically. The last step is obviously flawed, but they were hardly going to conclude that the brain processes information with electricity and neurotransmitters when the height of technology was the gear.
Nowadays, we know the physical substrate that the brain uses. We compare brains to computers even though we know there are no silicon microchips or motherboards with RAM slots involved. We do that because we figured out that it doesn't matter what a machine uses to compute; if it is Turing complete, it can compute exactly as much as any other computer, no more, no less.
> "Reminds me of autonomous vehicles a couple of years back".
I don't think any reasonable interpretation of "autonomous vehicle" includes the ability to change a tyre. My point is that sometimes hype becomes reality. It might just take a little longer than expected.
It has taken 10+ years to get to present day, from the start of the "deep learning revolution" around 2010. I vaguely recall Uber promising self-driving pickups somewhere around 8-10 years ago. A main difference between current AI systems and the systems behind the cyclical hype cycles ongoing since the 1950s is that these systems are actually delivering impressive and useful results, increasingly so, to a much larger amount of people. Waymo alone services tens of thousands of autonomous rides per month (edit: see sibling comment, I was out of date, it's currently hundreds of thousands of rides per month -- but see, increasingly), and LLMs are waaaaay beyond the grandparent's flippant characterization of "plausible-looking but incorrect sentences". That's markov chains territory.
But they aren't particularly autonomous, there's a fleet of humans watching the Waymos carefully and frequently intervening for the case where every 10-20 miles or so the system makes a stupid decision that needs human intervention: https://www.nytimes.com/interactive/2024/09/03/technology/zo...
I think Waymo only releases the "critical" intervention rate, which is quite low. But for Cruise the non-critical interventions was every 5 miles and I suspect Waymos are similar. It appears that Waymos are way too easily confused and left to their own devices make awful decisions about passing emergency vehicles, etc.
Which is in fact consistent with what self-driving skeptics were saying all the way back in 2010: deep learning could get you 95% of the way there but it will take many decades - probably centuries! - before we actually have real self-driving cars. The remote human operators will work for robotaxis and buses but not for Teslas.
(Not to mention the problems that will start when robotaxis get old and in need of automotive maintenance, but the system didn't have any transmission problem scenarios in its training data. At no time in my life has my human intelligence been more taxed than when I had a tire blowout on the interstate while driving an overloaded truck.)
If this is the end result, this is already a substantial business savings.
What "critical" intervention rate are you talking about? What network magically supports the required low latencies to remotely respond to an imminent accident?
How does your theory square with events like https://www.sfchronicle.com/sf/article/s-f-waymo-robotaxis-f... that required a service team to physically go and deal with the stuck cars, rather than just dealing with them via some giant remotely intervening team that's managed to scale to 10x rides in a year? (Hundreds of thousands per month absolutely.)
Sure, there's no doubt a lot of human oversight going on still, probably "remote interventions" of all sorts (but not tele-operating) that include things like humans marking off areas of a map to avoid and pushing out the update for the fleet, the company is run by humans... But to say they aren't particularly autonomous is deeply wrong.
I would be interested if you can dig up some old skeptics, plural, saying probably centuries. May take centuries, sure, I've seen such takes, they were usually backed by an assumption that getting all the way there requires full AGI and that'll take who knows how long. It's worth noticing that a lot of such tasks assumed to be "AGI-complete" have been falling lately. It's helpful to be focused on capabilities, not vague "what even is intelligence" philosophizing.
Your parenthetical seems pretty irrelevant. First, models work outside their training sets. Second, these companies test such scenarios all the time. You'll even note in the link I shared that Waymo cars were at the time programmed to not enter the freeway without a human behind the wheel, because they were still doing testing. And it's not like "live test on the freeway with a human backup" is the first step in testing strategy, either.
I was being vague - Waymo tests the autonomous algorithms with human drivers before they are deployed in remote-only mode. Those human drivers rarely but occasionally have to yank control from the vehicle. This is a critical intervention, and it seems like the rates are so low that riders almost never encounter a problem (though it does happen). Waymo releases this data, but doesn't release data on "non-critical interventions" where remote operators help with basic problem solving during normal operations. This is the distinction I was making and didn't phrase it very clearly. I think those people are intervening at least every 10-20 miles. And since those interventions always involve common-sense reasoning about some simple edge case, my claim is that the cars need that common-sense reasoning in order to get rid of the humans in the loop. I am not convinced that there's even enough drivers in the world to generate the data current AI needs to solve those edge cases - things like "the fire department ordered brand new trucks and the system can't recognize them because the data literally doesn't exist."
> First, models work outside their training sets.
This is incredibly ignorant, pure "number go up" magical thinking. Models work for simple interpolations outside their training data, but a mechanical failure is not an interpolation, it's a radically different change which current systems must be specifically trained on. AI does not have the ability to causally extrapolate based on physical reasoning like humans. I had never experienced a tire blowout but I knew immediately what went wrong, relying on tactile sensations to determine something was wrong in the rear right + basic conceptual knowledge of what a car is to determine the tire must have exploded. Even deep learning's strongest (reality-based) advocates acknowledge this sort of thinking is far beyond current ANNs. Transformers would need to be trained on the scenario data. There are mitigations that might work: simply coming to a slow stop when a separate tire diagnostic redlines, etc. But these might prove bitter and unreliable.
> Second, these companies test such scenarios all the time.
No they don't! The only company I am aware of which has tested tire blowouts is Kodiak Robotics, and that seemed to be a slick product demo rather than a scientific demonstration. I am not aware of any public Waymo results.
I'd also like to challenge people to actually consider how often humans are correct. In my experience, it's actually very rare to find a human that speaks factually correctly. Many professionals, including doctors (!), will happily and confidently deliver factually incorrect lies that sound correct. Even after obvious correction they will continue to spout them. Think how long it takes to correct basic myths that have established themselves in the culture. And we expect these models, which are just getting off the ground, to do better? The claim is they process information more similarly to how humans do. If that's true, then the fact they hallucinate is honestly a point in their favor. Because... in my experience, they hallucinate exactly the way I expect humans to.
Please try it, ask a few experts something and I guarantee you that further investigation into the topic will reveal that one or more of them are flat out incorrect.
Humans often simply ignore this and go based on what we believe to be correct. A lot of people do it silently. Those who don't are often labeled know-it-alls.
Implementation detail that will be solved as the price of AI training decreases. Right now only inference is feasible at scale. Transformers are excellent here since they show great promise at 'one shot' learning meaning they can be 'trained' for the same cost as inference. Hence the sudden boom in AI . We finally have a taste of what could be should we be able to not only inference but also train models at scale.
Is it going to scale to "superintelligence?" Is it going to be "the last invention?" I doubt it, but it's going to be a big deal. At the very least, comparable to google search, which changed how people interact with computers/the internet.
LLMs, irrespective of how powerful, are all subject to the fundamental limitation that they don't know anything. The stochastic parrot analogy remains applicable and will never be solved because of the underlying principles inherent to LLMs.
LLMs are not the pathway to AGI.
So what exactly is the usefulness of this discussion? You think "I'll trust my gut" is a useful argument in a debate?
FWIW my gut happens to agree with yours.
It hurts the pride of technical people that there's a revolution going on that they aren't involved in. Easier to just deny it or act like it's unimpressive.
How can I account for the cynicism that's so common on HN? It's got to be a psychological mechanism.
I thought a forum of engineers would be more interested in the practical applications and possible future capabilities of LLMs, than in all these semantic arguments about whether something really is knowledge or really is art or really is perfect
Hard disagree. LLMs merely present the illusion of knowledge to the casual observer. A trivial cross examination usually is sufficient to pull back the curtain.
Repeatedly, we’ve thought that humans and animals were different in kind, only to find that we’re actually just different in degree: elephants mourn their dead, dolphins have sex for pleasure, crows make tools (even tools out of multiple non-useful parts! [1]). That could be true here.
LLMs are impressive. Nobody knows whether they will or won’t lead to AGI (if we could even agree on a definition – there’s a lot of No True Scotsman in that conversation). My uneducated guess is that that you’re probably right: just continuing to scale LLMs without other advancements won’t get us there.
But I wish we were all more humble about this. There’s been a lot of interesting emergent behavior with these systems, and we just don’t know what will happen.
[1]: https://www.ox.ac.uk/news/2018-10-24-new-caledonian-crows-ca...
LLM proponents seem unwilling to accept that we comprehend the words we speak/write in a way that LLMs are not capable of doing.
Maybe their salary depends on them not understanding it.
That effective theory is knowledge, literally.
People harping about “stochastic parrot” are just people repeating a shallow meme — ironically, like a stochastic parrot.
LLM models are very far off from humans in reasoning ability, but acting like most of the things humans do aren't just riffing on or repeating previous data is wrong, imo. As I've said before, humans been the stochastic parrots all along.
No it isn't. The previous state of the art was markov chain level random gibberish generation. What OP described is an enormous step up from that.
Why? Text training data is already exhausted.
Next focus will hopefully be on reasoning abilities. Probably gonna take another decade and a similar paper to attention is all you need before we see any major improvements...but then again all eyes are on these models atm so perhaps it'll be sooner than that.
Literally today I used Bing and it was making up API parameters.
Code example looked fine, but didnt reflect reality.
The fault lies with humans using AI for something sensitive, without having the AI pass through certification etc. Part of the problem is glacial pace of laws around things, but that's nothing new isn't it; us humans being whiny, argumentative, inefficient, emotional meat bags about every little thing. I wonder, once we do make AGI, if it will wonder why it took us so damn long to tax the disgustingly wealthy, implement ww public healthcare, UBI, etc and solve the housing crisis by gasp building more houses...
We evolved, so our deep, deep underlying motivations pretty much always circulate around self-preservation and reproduction (resource contention).
The winter is going to be warm because of all the heat generated by GPUs ;)
global warming is killing us all
And.its not just incorrect sentences it's weird questions which are getting answered a lot better than ever before.
Why are you so dismissive? Have you ever talked or wrote with a computer which felt anything like a modern LLM? I have not
LLMs don’t have any ability to choose to update their policies and goals and decide on their own data acquisition tasks. That’s one of the key needs for an AGI. LLM systems just don’t do that / they are still primarily offline inference systems with mostly hand crafted data pipelines offline rlhf shaping etc…
There’s only a few companies working on on-policy RL in physical robotics. That’s the path to AGI
OpenAI is just another ad company with a really powerful platform and first mover advantage.
They are over leveraged and don’t have anywhere to go or a unique dataset.
For instance, a number of my conversations with ChatGPT contain messages it attempted to steer its own future training with (were those conversations to be included in future training).
That'll be investor types who bring this stupid "winter" on, because they run their lives on hype and baseless predictions.
Technology types on the other hand don't give a shit about predictions, and just keep working on interesting stuff until it happens, whether it takes 1 year or 20 years or 500 years. We don't throw a tantrum and brew up a winter storm just because shit didn't happen in the first year.
In early 2022 there was none of this ChatGPT stuff. Now, we're only 2 years later. That's not a lot of time for something already very successful. Humans have been around for several tens of thousand years. Just be patient.
If investors ran the show in the 1960s expecting to reach the moon with an 18 month runway, we'd never have reached the moon.
Like, are we using entirely different products? How are we getting such different results?
A lot of the rest of the world are using it for other things. And at these other things, the results are less impressive. If you've had to correct a family member who got the wrong idea from whatever chat bot they asked, if you've ever had to point out the trash writing in an email someone just trusted AI to write on their behalf before it got sent to someone that mattered, or if you've ever just spent any amount of time on twitter with grok users, you should be exceptionally and profoundly aware of how unimpressive AI is for the rest of the world.
I feel we need less people complaining about the skepticism on HN and more people who understand these skeptics that hang out here already know how wonderful a productivity boost you're getting from the thing they're rightly skeptical about. Countering with "But my code productivity is up!" is next to useless information on this site.
I appreciate your anecdotes on failures/embarrassment for people outside of tech- there's pretty clearly a gap in experience, understanding, and marketing hype.
I don't think it's useless to ask what that gap is, and why GP got such poor results.
I have no idea about AGI but honestly how can you use claude or chatgpt and come away unimpressed? It's like looking at spaceX and saying golly the space winter is going to be harsh because they haven't gotten to Mars yet.
Mars is hard but there are paths forward. More efficient engines, higher energy density fuels, lighter materials, better shielding, etc, etc. It's hard but there are paths forward to make it possible with enough time and money. We have an understanding of how to get from what we have now to what makes Mars possible.
With LLMs, there is no path from LLM -> gAI. No amount of time, money or compute will make that happen. They are fundamentally a very 'simple' tool that is only really capable of one thing - predicting text. There is no intelligence. There is no understanding. There is no creativity or problem solving or thought of any kind. They just spit out text based on weighted probabilities. If you want gAI you have to go in a completely different direction that has no relationship with LLM tools.
Don't get me wrong, the work that's been done so far took a long time and is incredibly impressive, but it's a lot more smoke and mirrors than most people realize.
And "LLM's just make plausible looking but incorrect text" is silly when that text is more correct than the average adult a large percentage of the time.
IMO it will be done vertical by vertical, with no standard interface coming for a while.
But they lived in grass huts and the highest they had ever been off the ground was when they climbed a tree.
One day a genius was born on the island. She built a structure taller than the tallest tree. "I call it a stepladder," she said. The people were amazed. They climbed the stepladder and looked down upon the treetops.
The people proclaimed "All we have to do now is make this a little higher and we can reach the moon!"
A mere 5 years ago I was firmly on the camp (alongside linguists mainly) that believed intelligence requires more than just lots of data and a next token predictor. I was clearly wrong and would have lost a $1000 bet it I had put my money where my mouth was back then. Anyone not noticing how far things have come I think are mostly moving goalposts and falling to hindsight bias.
A better analogy is that the genius person in the village built a step ladder made of carbon nanotubules. Some people proclaimed "All with have to do is keep going and we can reach the moon with a very tall ladder!' Other people - many quite smart - proclaimed reasonably: "This is impossible. You are not realizing the unique challenges and materials we have not yet researched we need to build something like that."
Some in society kept building. The ladder kept getting higher. They run into issues like oxygen and balance so the ladder is redesigned into an elevator. Challenges came and were thought insurmountable until redesigns were still found to work with the miraculous carbon nanotubule material which seemed like a panacea for every construction ill.
Regardless of how high the now elevator gets and regardless of how many times the elevator gets higher than what the naysayers firmly believed is impossible the same naysayers keep saying they will never get much higher.
And higher the elevator grows.
Eventually a limit is reached, but that limit ends up being far higher than any naysayers ever thought possible. And when the limit is reached the naysayers all gathered and said "told you this would be the limit and that it was impossible!'
The naysayers failed to see the carbon nanotubules for the revolutionary potential that it had, even if they were correct that it wasn't enough.
And little did everyone know that their society was mere months ago from another genius being born that would give them another catalyst on the order of carbon nano tubules that would again lead to dramatic unexpected and long-term gains to how high the elevator can grow.
The difference is current ML already has real use cases right now in its current form. Some examples are OCR, text to speech, speech to text, translation, recommendations (for eg. Facebook Tiktok etc.) and simple NLP tasks ("was [topic] mentioned in the following paragraph"). Even if AGI is proved impossible, these are real use cases that hold billions in value. And ML research is also considered a prestigious and interesting field academically and that will likely not change even if investors give up on funding AGI.
You missed the point of the parent comment's post. He's talking about the current post chatbot GenAI hype (i.e., the massive amounts of funding being poured into companies specifically after this turning point).
1. You don't need massive amounts of funding to work on ML. A good deal of important work in ML was done in universities (eg. GANs, DPM, DDPM, DDIM) or were published before the hype (Attention). The only qualifier here is that training cost a lot right now. Even so, you don't need billions to train and costs may go down as memory costs come down and hardware competition increases.
2. You don't need VC type investors to fund ML research. Large tech companies like Facebook, Google, Microsoft, ByteDance and Huawei will continue investing in ML no matter what, even if the total amount they invest goes down (which I personally don't think it will). Even if they shift away from chatbots and only focus on simpler NLP tasks as described above, related research will still continue as all these tasks are related. For example, Attention was originally developed for translation and Llama 3.2 isn't just a chatbot and can also do general image description, which is clearly important to Facebook and ByteDance for recommendations and to Google for image search and ads. Understating what people like and what they are looking at is a difficult NLP problem and one that many tech companies would like to solve. And better image descriptions could then improve existing image datasets by allowing better text-image pairs, which could then improve image generation. So hard NLP, image generation and translation are all related and are increasingly converging into single multimodal LLMS. That is, the best OCR, image generation translation etc. models may be ones that also understand language in general (ie. broad and difficult NLP tasks). The issue is that OP assumes it must be AGI or bust.
It really feels like a substantive step forward in terms of computer utility kind of like spreadsheets, databases, apps. We'll see how far it takes us down the line of human replacement though.
Definitely in the coming decade, we can prepare for a lot of the simpler tasks in an office to be taken over by AI. There are plenty of scenarios in which someone is managing a spreadsheet because an SME doesn't have the money to hire developers to automate & maintain that process - with advanced LLMs they can get it done by asking it to.
It will likely take well over a decade to recoup investment on any new chip fab at this point. Chasing ~4 customers on one narrow use case is nonsensical from the perspective of anyone running these companies.
If he wants to actually accomplish his objectives, he needs to get inside the minds of the executives that run these companies. Empathize with their concerns and then develop a strategy for walking them towards a path to build even one additional factory.
Throwing out a cartoonish figure and then hoping to be taken seriously is not something I'd expect from the CEO of something so adjacent.
Empathy is a good practice when managing others and have control over what they do, but in business negotiations it is often productive to just make your wants and budget clear.
This is actually quite clever phrasing. "A few thousand days" is about ten years, assuming normal usage of 'few' (ie usually a number between 3 and 6 inclusive).
Now, if you, as a tech company, say "X is ten years away", anyone who has been around for a while will entirely disregard your claim, because forward-looking statements in that range by tech companies are _always_ wrong; it's pretty much a cliche. But phrasing as a few thousand days may get past some peoples' defences.
And the mistake isn't thinking more generally about 'the solution to AGI'.
The mistake is thinking about 'AGI'.
There will never be an artificial general intelligence. There will never artificial intelligence, full stop.
It's a fun concept in science fiction (and earlier parallel concepts in fantasy literature and folk tales). It's not and will never be reality. If you think it can be then either you are suffering from 'science fiction brain' or you are a fraud (Sam Altman) or you are both (possibly Sam Altman again).
It's really hard to keep good employees working at these places for a long duration. Innovation is something your customers do. Your job is to make it as boring as fucking possible.
I could only stand it for 3 years before I had to quit (Samsung). I know others who still enjoy the experience though.
- The public focus on AGI is almost a distraction. By the time we get to AGI highly-specialised models will have taken jobs from huge swaths of the population, SWE and CS are already in play.
- That AI will need to carry out every task a role does to replace it. I see this a lot on HN. What if SWEs get 50% more efficient and they fire half? That's still a gigantic economic impact. Even at the current state of the art this is plausible.
- The notion that everyone laid off above will find new employment from the opportunities AI creates. Perhaps it's just a gap in my knowledge. What opportunities are so large they'll make up for the economies we're starting to see? I understand the inverting population pyramid in the Western world helps this some also (more retirees/less workers).
On the third point, I think we've always seen this happen even in massive shocks like the Industrial Revolution (and the Second Industrial Revolution with assembly lines etc. and the Computer Age)
It might be hard for people to retrain to whatever the new opportunities are though. Although perhaps somewhat easier nowadays with the internet etc.
Personally, preindustrial life sounds pretty rough, but its all just apples and oranges! The future will continue to happen, to critique the present and how we got here is not to exhort the past (unless, you know, you are a particularly conservative person I guess).
This Economist article talks about how looking at the historical data, historians now see that the height of the average Englishman actually went down due to malnutrition. https://web.archive.org/web/20210905065401/https://www.econo...
The myth that the Industrial Revolution was a wonderful time is just that, a myth. The actual reality of the AI revolution will likely be the same. Record number of billionaires and record number of people in deep poverty at the same time.
And I agree with you. Further, the argument about economic prosperity isn’t equal for everyone. And increased worker efficiency isn’t directly (or sometimes at all) linked to worker satisfaction or even increased wages.
However, generally I think being a software developer might be not a career in 10 years which is terrible to think about. Designer too. And all of this is through stealing peoples work as their own.
For image models, people are often pretty surprised to learn that they are only a few gigabytes in size, despite training on petabytes of images.
If it's called "bank-coin"; "swedish crowns" or "bitcoin" it doesn't matter because it's all digital anyway.
On that count, the technological innovation is here, but it's centralised in a few "trusted" entities, just like everything.
Not everything has this trajectory, and it took 10 years more than expected. But it's coming.
Not saying AI will be the same, but underestimating the impact of having certain outputs 100x cheaper, even if many times crappier seems like a losing bet, considering how the world has gone so far.
So yes: handling 100,000 passengers is a milestone. The growth from 10,000 to 100,000 implies it’s going to keep growing exponentially. But eventually they’re going to encounter stuff like Midwest winters that can easily stop progress in its tracks.
"People in San Francisco tag a driverless car"
https://www.reddit.com/r/CrazyFuckingVideos/comments/1fqcpq2...
About driverless cars, new tech adoptions often start slow, until the iceberg tips and then it's very quick change. Like mobile phones today.
I remember thinking before smartphones that had entire-day battery and good touchscreens: These people really think population will use phones more than desktop computers? Here we are.
The cars aren't making all their decisions in real-time like a human driver. They, Waymo, meticulously mapped and continue to map every inch of the traversable city. They don't know how to drive, they know how to drive THERE.
It would be like if I went to the DMV to take a driving test. I would fail immediately, because the parking lot is not one I've seen and analyzed before.
"true" self driving is not possible with our current implementation of automobiles. You cannot safely mix automobiles that self-drive with human drivers. And the best solution is to converge towards known routes. We don't even necessarily how to program the routes - we can instead encode them in the road itself.
It might occur to you that I'm speaking about rail. The reality is it's trivial to automate rail systems, but the variables of free-form driving can't be automated.
In the second case there are inherent technical challenges to using non-fiat currency and the fx volatility with fiat is wild. There are also barriers and inconveniences to conversion. With GPT writing code, the user can review for quality and still be many x more productive and there is far fewer fees and risk of loss.
It's risky to take two failed or slow innovations and assume that all innovations will be failed or slow.
Zero sum game or fixed lump of work fallacy. Think second order effects - now that we spend less time repeating known methods, we will take on more ambitious work. Competition between companies using human + AI will raise the bar. Software has been cannibalizing itself for 60 years, with each new language and framework, and yet employment is strong.
New product that push that bar and can command a decent margin (and good staff salaries) as long as there's a business case/demand and feature-sets that currently command a decent margin will be available for dirt cheap prices (managed by one or two person outfits).
Your comment really got me thinking, it's time to upskill haha. Aside from biotech and robotics do you see any areas particularly ripe for this push?
A lot of the predictions of job loss are predicated on an unspoken assumption that we’re sitting at “task maximum” so any increase in productivity must result in job loss. It’s only true if there is no more work to be done. But no one seems to be willing or able or even aware that they need to make that point substantively—to prove that there is no more work to be done.
Historically, humans have been absolutely terrible at predicting the types and volumes of future work. But we’ve been absolutely incredible at inventing new things to do to keep busy.
They’d be excited at getting 100x of 100% output out an AI for 20 dollars a month and laying you off as redundant. If you aren’t scared of the potential of this technology you are lying to yourself.
Why should I be scared of technology that makes me more productive?
If they’re happy with current rate of progress (and in many companies that is the case), then a productivity increase of 100% means they need half the current number of engineers.
And the “current rate” is competitively defined. So if AI can make software developers twice as productive, then the acceptable minimum “current rate” will become 2x faster than it is today.
Ai is trending towards a point where it makes your employers productive such that they don’t need you.
Yet people are still employed today. They are doing the many new jobs that the productivity boost of digital computing created.
Productivity creates jobs.
The computer created jobs because it takes human skills to talk to the computer.
It takes very little skill to talk to an LLM. Why would your manager ask you to prompt an LLM to do something for you when he can do it himself? You going to answer this question with another analogy?
Just think reasonably and logically. Why would I pay you a 300k annual salary when a chatGPT can do it for nothing? It's pretty straightforward. If you can't justify something with a straightforward answer, likely you're not being honest with yourself.
Why don't we use actually evidence based logic to prove things rather then justify things by leaping over some unreasonable gap with some analogy. Think about the current situation, don't base your hope on a past situation and hope that the current situation will be the same because of analogy.
Now you might say AI means that I will run out of things that my employer needs me to do. And I'll repeat what I said above: you've got to prove that. I'm not going to take it on faith that you have sussed out the complete future of business.
What we can do is make a logical and theoretical extrapolation. If AI progresses to the point where it can do every single task you can do in seconds, what task is there for you left to do? And how hard is the task? If LLMs never evolve to the point where they can clean toilets, well then you can do that, but why would the boss pay you 300k to clean the toilet?
These are all logical conjectures on a possible future. The problem here is that if AI continues on the same trendline it's traveling on now I can't come up with a logical chain of thought where YOU or I keep our 300k+ engineering jobs.
This is what I keep hearing from not just you, but a ton of people. That analogy about how technology only created more jobs before with no illustration of a specific scenario of what's going on here. Yeah if LLMs replace almost every aspect of human intellectual analysis, design, art and engineering what is there left to do?
Clean the toilet. I'm not even kidding. We still have things we can do but the end comes when robotics catches up and is able to make robots as versatile as the human form. That's the true end when the boss has chatGPT clean the toilet.
They produce twice as much and grow.
What forum is this???
You're relying on theoretical work needed by employers to be unlimited. You're also assuming all of this additional work can't be handled by an LLM.
First of all fixed lump of work is not a fallacy. We do know there is a limit as there's limits in the amount of work human brains can even comprehend. A limit exists. We don't know where exactly this limit is, but a limit DOES exist and an LLM may possibly cover that limit.
Second, you have to assume that this "additional work" can't be handled by the LLM. How can you be sure? Did you think about what this work actually is? My first thought was "cleaning the toilets."
>What forum is this???
I assume it's a forum of people who don't base their lives off of concepts with buzzwords. “Fixed lump of work fallacy” is a fancy phrase for a fancy concept... that doesn't mean it's an actual fallacy or actually true. Literally you just threw that quote up there as if the slightly clever wording itself proves your point.
What Exactly is this additional work that will pop up once LLMs are around and so powerful they can do all human intellectual work? Can you even do a concrete/solid real-world analysis without jumping to vague hypotheticals covered by fancy worded conceptual quotations? The last guy used analogies as part of his logical baseline of reasoning. Wasn't convincing to me.
If they're high growth yes, if they're in the majority of businesses that are just trying to maximise profit with negligible or no growth then likely not.
It's really only AGI, by eclipsing human capabilities across all useful work, that breaks this dynamic and creates the prospect of permanent structural unemployment.
We need to come up with ways of providing meaningful roles for the large percentage of people whose peg shape doesn't fit the median job hole.
Not all humans are going to be capable of this as time goes on. Those humans will be subjected to abject poverty.
The reality is not everyone can possess a highly skilled knowledge job. Not everyone can go to college. What of them?
Now, initially this still looks like it's going to reduce demand for janitors by 99%. So it's still going to cause mass unemployment, right? Except, it's going to substantially reduce the cost of janitorial services, so more will be purchased. Not just janitorial services, of course. We'll deploy such robots to do many things at higher intensity than we do today, and as well as many things that we don't do at all right now because they're not cost effective. So in equilibrium (again, the transition may be messy), with 99% automation we end up with an economy 100x the size, and about the same number of humans employed.
I know this sounds crazy, but it's the historical norm. Today's industrialized economies already have hundreds of times the output of pre-industrial economies, and yet humans mostly remain employed. At no point did we find that we didn't want any more stuff, actually, and decide to start cashing out productivity increases as lower employment rather than more output.
This worked before because commonly people couldn't even read or do basic math. We figured that out and MUCH more and now everyday people are taught higher think for many years. People, today, are extremely smart as compared to all of human history.
But IMO we've kind of reached a ceiling. We can't push people further than we already have. In the last two decades this became very evident. Now almost everyone goes to college, but not all of them make it through.
The low-end has been steadily rising, that now for 20 bucks an hour you need a degree. That's with our technology NOW. We're already seeing the harmful effects of this as average or below-average people struggle to make even low incomes.
It's true that humans will always find new stuff to do. The issue is as time goes on this new stuff goes higher and higher. We can only push humans, as a whole, so far.
Think how many places you see shitty software currently.
My wife was just trying to use an app to book a test with the doctor - did not work at all. The staff said they know it doesn’t work. They still give out the app.
We are surrounded by awful software. There’s a lot of work to do- if it could be done cheaper. Currently only rich companies can make great software.
Lots of the awful software is made by awfully rich companies - and lots of good software is made by bootstraped devs.
To mention some interesting examples, both Amazon and Google has gone from great to meh soon after they went from startups to entrenched market leaders.
I guess this is why I’m excited. AI will give smaller motivated teams a lot more firepower. One committed person can (or may soon be able to) take on the might of a big company.
These companies are making crap software because their scale makes them hard to compete with. They know there’s no other good options.
I think Sam Altman’s right that there’ll be a 1 person unicorn company at some point.
Oh, not just smaller shops, I have many issues with Android and other Google products -- from bugs to things that just don't work that have existed for decades, and there is no action on those over the years. Surely Google has the resources? Right? riiight?
This is a human problem, not a technology problem.
This is kinda ironic in a thread that's basically about the AI hype landscape, but you've just reduced the amount of SWE "power" your example organization has there by 25%.
You know what happened last time we got 50% more efficient? It was when github and npm arrived. LLM are saving time and making us more efficient, but that's peanuts compared to the ability to just “download a lib that does X” instead of coding this shit on your own. And you know what happened after that? SWE position skyrocketed.
With that said it's a gamble since somebody might come along with a $10MM model based on some new technique and then OpenAI's cost structure becomes a problem. Presumably their scientists could adapt pretty quickly though if that happened.
That sounds exactly like venture-funded startup thinking to me.
Just because there’s an anti-Altman/-OpenAI sentiment on HN at the moment, it doesn’t mean that he’s wrong about everything, nor the people at TSMC are necessarily right.
Where does Altmann get $7 trillion dollars
Again, the Russians laughed at Musk. The legacy space companies didn’t produce reusable boosters when they clearly could’ve, and people laughed at the idea until it was working. Etc.
(I’m not sure what relevance the NASA comment has; of course small underfunded startups need money to realise their goals? SpaceX needed NASA’s money to develop various projects faster, and NASA needed SpaceX to a) reduce launch costs and b) reduce their reliance on Russia for human launches.)
Who will by pay $7 billions so Altman can follow his?
SpaceX costs were much lower and there goal was to become cheaper.
OpenAI will cost more even with all the chips Altman wants. Those chips need energy.
Sam Altman - savior of humanity.
HN comments: https://www.youtube.com/watch?v=ATCrco_V-24
if this house of cards collapses, it will be magnitutes bigger than any bubble (be it hype or fraud) we've seen so far.
what a waste of resources and focus, i really can't take vc players seriously any more (from what is left after the crypto hype run)
That's what you see. When you're out of earshot we refer to you by several infantilizing expressions that connote npcness.
AMD and NVidia are important now because someone else struck gold on their property. Within 5 years they won't be because both are fucking up monumentally for different reasons: AMD for being AMD [0] and NVidia for expecting 10x markup to continue indefinitely.
People hate success but LLM swarms are the next wave of improving output and they consume compute on par with training for inference.
[0] https://twitter.com/__tinygrad__/status/1839219312952766942
What would have been the alternative for them? Excluding the US, software is not a high margin business.
So a yearly loss of 4 billion dollars or losing $10 million daily. The IPOs basically (mostly) have been operating just the way Bitcoin has been - Next bigger fool theory. That you pass on the potato to the next bigger fool till it ends to the last in the chain like Twitter who doesn't know what to do to churn out those inflated sums paid along the chain.
Not sure how many days OpenAI can afford to lose 10 million dollars daily but current macroeconomic environment, prospects aren't great.
Also, revenue you can't take home. Doesn't matter how many billions.
It is the profit that matters in the end.
These things impress people who fail to digest business fundamentals. Anyone with a suit and a slick powerpoint preso can in theory start a money-incinerating machine. The real measurement is the profit. And before someone makes the "capture the market" comment - this isn't ZIRP anymore. Money costs fuckin money now.
What OpenAI is doing is "sustainable" if they keep growing like that. From summer 2023 to summer 2024 they 6x their revenue to 3.4B USD. If they 4x next year and 3x the year after, they would make 40B in revenue.
And lo and behold, ClopenAI has already hired CHIPS act people:
"To bolster its efforts, OpenAI has hired Chris Lehane, a Clinton White House lawyer, as its vice president of global policy, along with two people from the Commerce Department who worked on the CHIPS Act, a bipartisan law designed to increase domestic chip manufacturing. One of them will manage future infrastructure projects and policy."
We'll build plants overseas if you don't give us CHIPS Act money is a great scheme.
Such a misguided take.
If it were that easy to build fabs to compete with TSMC, Intel would do it. China would do it.
Hardware is way harder than software.
https://www.pcworld.com/article/2355435/intels-lunar-lake-is...
2) that's not Intel's latest chips, these are: https://www.intel.com/content/www/us/en/products/details/pro... https://www.intel.com/content/www/us/en/products/details/pro...
One fully made by Intel Foundry, the other with at least some considerable TSMC parts (which isn't unexpected for the market segment it's in and the fact it was allegedly designed by Habana before it was bought out).
Lunar Lake is the codename for Core Ultra Series 2 mobile processors designed by Intel, released on September 24, 2024.
Manufactured by: TSMC
It's September 27 round abouts here so I'm not sure it gets more recent than that.
2) It does get more recent than that: Granite Rapids (Xeon 6 built fully on Intel processes) and Gaudi 3 (TSMC built) were announced on the 24th and will be released later this year.
Maybe you shouldn't get so sassy when discussing things you know very little about.
> ...and will be released later this year
> ...which completely disproves your lie about Intel sending CHIPS money to TSMC
Also, you're arguing a strawman here. What I wrote:
> US is already throwing CHIPS dollars at Intel
They didn't receive any money from CHIPS Act yet, did they?
It's just an insane risk to take on.
The CHIPS act has provisions to help fund fabs in allies.
This is why Biden and Modi announced a Feb dedicated to US and Indian defense systems [0] at the QUAD summit as well as as the designation of the UAE as a "Major Defence Partner" along with India [1] which includes tech transfer conditions.
A significant portion of the CHIPS and IRA is set aside to help with subsidizing international allies tech and innovation ecosystems in order to ensure they don't lean towards China [2]
[0] - https://www.bloomberg.com/news/articles/2024-09-23/biden-mod...
[1] - https://www.reuters.com/world/us/harris-plans-raise-gaza-cea...
[2] - https://cset.georgetown.edu/publication/agile-alliances/
For a number of commodity components, there was a heavy reliance on China due to low margins. Now there is a push to move those portions of the supply chain away to other partners.
Population size is about to peak. Up until now, for as long as we know, it has been growing. Starting at the latest with colonisation, we've had more people, more resources, new markets advancing into buyers of new products. Once societies advance to a certain point, they begin to shrink, this is well studied.
Without these growth factors, does it seem likely we'll see something as transformative as the automobile or the internet again?
Possibly bleak and badly informed, but I find it plausible to think that the party is about to end. Most of us here have probably seen what happens to a company when they stop growing. Spoiler: It's typically not innovation.
But that's not really growth, it's optimisation. That is exactly the kind of thing that a company that stopped growing does. That wouldn't necessarily make it "the next big thing" though, in the "new frontier" ways we've seen in the past.
And he was tech savy, building circuits, computers etc, he just didn't think it would be reliable or better than what existed... definitely a vinyl guy.
Didn't pan out well for me.
Not that they were great options, but there was alot of existing media available, and hardware was common, so an expensive bulky device to play fragile media at lower quality than vinyl, was how it was viewed by many initially.
Edit: just checking, the Sony Walkman came out in 1979, CDs in 1982, but the Walkman dominated the 80s & 90s, so its easy to see why CDs were dismissed
I guess in terms of storage I can't quite grasp why he thought that CD wouldn't take on. (Hindsight 50/50, of course.)
0: https://en.wikipedia.org/wiki/Cassette_tape#Data_recording
So you end up with a whole bunch of poor overweight people who need 4500 cals a day to sustain their mass, reporting that they have a hard time sustaining their diet. Obesity is a huge problem in lower income demographics, the same demographics that report high food insecurity.
I also don’t see think it’s historically accurate. My understanding was that widespread electrification were motivated by lighting, the radio, and other applications. Getting electricity to more people probably helped spur further development, sure, but there were already “killer apps.”
Part of the AI scam peeling back the facade on every AI product and realizing "oh this is actually pretty useless unless we already have high-quality automated data systems internally at $ORG", and the advancements in AI only make that nominally more accessible. No, synthetic data is not the solution, and Copilot won't build you a robust data platform.
Digitalization and digital literacy are long-term trends that AI is at the whim of, but pretends otherwise.
a) Given how geopolitical semiconductors and AI are right now. Why he is running around the world running his mouth seemingly without the cooperation of the US government.
b) What does Microsoft have to say about OpenAI building a parallel cloud.
c) OpenAI has zero competence in designing and building their own end to end stack but yet they are going to jump straight to $7T worth of infrastructure. Even Apple acquired PA Semiconductor and a number of smaller startups to build out their team.
As an American citizen you don't need to cooperate with the US government to talk to other people.
Rare exceptions are old companies where the founder is still around and rejects deal after deal on pride. OpenAI is nothing like that, as the profit/non-profit drama exemplifies.
Take Anthropic for a more legible example: Amazon and Google both gladly bought large stakes in it.
To give less reasons for people and governments to accuse them of monopolistic behaviour. Maybe. Wild speculations from my end.
https://www.nytimes.com/1992/01/07/science/2-experts-say-wom...
But it was just two people and they were criticized by peers for being space cadets at the time. There are always some people ready to make a fool of themselves for recognition, perhaps.
Let's talk about that. GPT-3.5 (specifically text-davinci-003) was a massive leap over everything that came before it. GPT-4 was a massive leap over GPT-3.5. Everything since (GPT-4 Turbo, GPT-4o, GPT o1) feels like steady incremental improvement (putting aside multimodal capabilities) rather than massive leaps. I'm far from convinced that the rate of progress made with foundation models in 2022-2023 has continued in 2024 let alone from extrapolating that it will continue for the next several years.
Comparing the original mission as a nonprofit researching AI safety vs. a for-profit demanding the planet pour trillions of resources into its growth... Wild.
I've been in semiconductor / chip design for almost 30 years at 8 different companies including 2 that owned their own fabs. I've also worked on 2 different AI accelerator chips.
Altman sounds like someone who has no idea about the industry. The numbers in the article are laughable.
This quote summed it up.
> It is still unclear how all this would work. OpenAI has tried to assemble a loose federation of companies, including data center builders like Microsoft as well as investors and chipmakers. But the particulars of who would pay the money, who would get it and what they would even build are hazy.
He's trying to get a bunch of diverse companies including Microsoft to fund this but it's unclear what kind of chips he actually wants. Many of the big companies including Microsoft are designing their own custom AI chips.
But the article mentions Nvidia. Does OpenAI have any plans to design their own chips or do they just use Nvidia? It may be difficult to get Nvidia competitors to want to join his effort.
> TSMC makes semiconductors for Nvidia, the leading developer of A.I. chips. The plan would allow Nvidia to churn out more chips. OpenAI and other companies would use those chips in more A.I. data centers.
The question is whether LLMs are going to plateau. Either OpenAI and the tech industry system can keep progress going, or else more basic science is needed, in which case a government-project level of coordination is needed instead.
Steve Jobs actually knows a lot more about many things than most people realise.
I think it also sums up a lot about what I am seeing from software developers in general. Very little understanding of hardware. And even more so about hardware industry itself.
HN itself is a very good example and evidence.
My nerd hype died as I used gpt4+ and Claude 3.5+ whatever for coding tasks and realized if a human somewhere in the training data hadn't already solved a programming problem/wrote code for a solution the LLM's are useless for said problem.
They're useful as an assistant/junior coder to do repetitive tasks/things you/other humans already have already done tho. BUT that's not going to revolutionize programming as you already have to know how to code to use them.
* and "hallucinations" should be referred to as "bullshitting" when it comes to LLM's as that's what it really is.
See my experience is the opposite, and gpt3 was able to answer questions about the niche Haskell library I wrote better than I was. Unless you posit that there's some secret cabal of people using this Haskell library and writing answers for GPT to consume somewhere in the depths of OpenAIs org structure, it's pretty obvious to me that LLMs are quite capable.
Blackwell cabinets are $3M each and have significant power requirements and thus you'd likely need to be selling your products at $100/month/seat to turn a slim profit.
But CEOs often practice a form of charismatic leadership whereby you have to project having more knowledge than you actually do. That is because their leadership is both less democratic and less secure than their counterparts. You've got to be constantly maneuvering against challenges to your legitimacy.
I think it has something to do with the Gemeinschaft–Gesellschaft dichotomy from German sociology. You might think that a tech company would be the quintessential form of Gesellschaft, but in reality, the community of tech CEOs and VCs is far more like a social club than it is like a rational, contractual relationship.
factors are that international markets rely less on local cults of personality and just the financial, it’s also fair to say that TSMC is risk averse, most amusingly is that these personality cults work out fine if you throw enough money at them so I can’t fault this podcast bro for trying
"The NYT claims it discussed OpenAI's negotiations with nine people close to the discussions but who wish to remain anonymous."
"Claims"? Generally newspapers do not lie about the sourcing for a story.
in retrospect, press media was much more critical of meta and the whole metaverse schtick. it took long but all the money poured on the research has brought more meaningful results with the orion prototype.
we are giving this AI company too much air time. let them do their bidding by themselves.
he is 'Michael jordan of listening' or that he knew within 3 mins that sam is next bill gates.
These are actual paul graham quotes not something from Silicon Valley TV show.
Edit: to ppl responding "you should already know know paul graham is a grifter" . I don't believe is the case given his essays about meritocracy.
> Bankman-Fried's first pitch meeting with the VC firm's partners during its Series B round is also included in the profile, in which the founder laid out his vision to investors for FTX to become a super-app — all while he was playing the video game "League of Legends" on his computer.
Even with the advancements made by OpenAI, some of his takes remind me of a Dollar Store Elon Musk, let alone his despicable way of doing business and seemingly complete lack of any morals.
To your average TSMC exec he's closer to a Jake Paul than to a Lisa Su or even Jensen Huang.
Like, bro, all the sovereign wealth funds of the Arab Gulf hold less than $3 trillion, mostly in illiquid assets that you could take a 50% haircut if you need to liquidate. And after the $100b Vision Fund fiasco where Saudi Arabia was subsidizing "Uber for Dog-walking" startups for years (no thanks to Masayoshi Son), I'm not sure they're going to be keen subsidizing yet another bubble.
But, isn't that the default in VC? Aggressive, toxic optimism because it's not your money and you need to promise otherworldly yield to get capital.
I know Sam is gonna be a billionaire assetwise soon but I wonder if it’ll make him more realistic or even more “idealistic” (bigger bullshitter)
I don’t want to live in a world where nobody takes risks.
> However, the latest OpenAI statements have rolled back such talk to "mere" hundreds of billions. It is reported that years of construction time would also be needed to satisfy the OpenAI compute scaling plans.
Yeah so what? It will take years to build so build 36 mega data-centers in parallel.
> the firm has an income of approximately $3 billion per year, which is put in deep shade by its $7 billion annual expenditure.
Those are tiny numbers. Airpods are a $14B/year line of products. OpenAI needs to be ready to spend 100B/year.
> Likewise, Apple launched its iPhone 16 and 16 Pro earlier in the month with a lot of talk about Apple Intelligence, but the first of these AI features won’t be available on the new devices until next month.
I’m on a 15 pro max with the 18.1 PUBLIC Beta. I have these features and so do 16 users. Mark Tyson. This is terrible lazy journalism. You add half true statements to make your point.
Remember when Blockbuster mocked Netflix?
Remember when Steve Ballmer mocked the iPhone?
Remember when Zuck was mocked for the Metaverse?
I'm pretty sure Sam would have heard similar mockings like "You can't take on Google behemoth, podcaster bro" before.
But the sad part is, the newer tech generation is so brainwashed that it takes the side of the mocker instead of the mockee.
Watch Sam partner with some other fab like Intel and make TSMC irrelevant. Of course HN will then side with Sam
One of these things is not like the others. You listed 3 things proven silly with hindsight, then you post one comment that is mere hopeful wishing.
Actually, he was responsible for the competitor that PayPal bought out because the market wasn't big enough for two companies, and was forced out of PayPal because he was pushing ruinously bad ideas.
He wasn't the marketing guy. He was the guy who got fired because his tenure as CEO was "I don't understand this Java stuff, and I want to contribute to the codebase, so we really need to rewrite it in Classic ASP", problematic on so many levels.
Musk made some amusing comments that he thought were insightful about this that showed how little he really knew:
He also suggested that PayPal should have written its front end in C++ (because C++ in 2000 was an excellent choice for front-end web development?!?), and later cited Blizzard writing WoW in C++ as proof that this was the correct decision that Max and the board never understood.
(Oh, and "Microsoft has a DLL library for anything you could possibly ever want to do, but you can't get Unix libraries for anything.")
He also said that Max "never really understood this" despite Musk's efforts to educate him, and that these decisions are also why PayPal "hasn't been able really add any new features".
He then complained about the Board's decision to fire him as making him "more careful about who he allowed to invest in his companies in the future": 1) he was the largest shareholder, but it was not "his" company, he only held a fraction of it, and 2) the idea wasn't even his! Really what he means here is "I want to make sure to hand-select a bunch of yes-men so I get what I want regardless of whose idea it is, how good the idea is, and what I want to do with it."
"Responsible" indeed.
Just ... OMFG ... I used to do this all the time? This used to be how you learned things? You mean I get to read 10 sites and 25 dumb answers to get to a possibly correct one? And all the while I got ads coming at me for this and that. A friend was trying to decode some chat convo we had about GLP-1 drugs and get factual information.
Google search was nothing but direct ads selling "IF YOU LIKE OZEMPIC YOU WILL LOOOOOVE NOZEMPIC - ITS ALL NATURAL!!" and the content-mill blogs and such that tout the benefits of some "NOZEMPIC" variant.
AI has changed the way I learn and research. I have a very capable tutor on demand for any subject there is, any time I want to use it. Its mostly right or "good enough for government work" as the old guys say.
I was once an AI hater - now Im an AI evangelist and Im turning people in my sphere of influence into AI users too. Once you show them the value prop they totally get it. Learning and such no longer requires manual work for the 90% case.
No. Google is really not useful anymore, it wasn't like that. It is IMPOSSIBLE to search for what you want, Google just think they know what you want and literally change the words to match. It is crazy.