Investors are happy to pay premium for tech, but not for AI
bloomberg.com
bloomberg.com
1980 after the foundations of neural networks, but it was too computationally intensive to be useful
2009 with Watson
https://www.hiig.de/en/a-brief-history-of-ai-ai-in-the-hype-...
That's some extreme cherry picking.
During that time period, the internet and smartphones alone have completely changed society (for better and worse) in the span of only three decades, despite the former going causing a minor economic crash in its infancy.
Almost everything is different except human nature. The scammers are innovating just like everyone else.
We still burn fossil fuels to a large extent, still drive but not fly cars, still live on Earth not in space, still die of the same causes, etc.
I watch a long cargo train that looks like it's form the 80s go by and wonder how much the internet changed cargo hauling. I'm sure with the logistics the internet made things a lot more efficient, but the actual hauling is not much different. It's not like we teleport things around now. You can order online instead of out of a catalog, but brick stores remain. You can read digital books, but still plenty of printed materials, bookstores, libraries.
I honestly think this overstates the case pretty severely. They have certainly caused societal change, but from what I can see, society as a whole is not actually all that different from what it was before all of that.
TBH, I'm not sure how to quantify housing bubbles either. I'd bet most of the country has much higher home prices now than in 2007. I bet they were higher than 2007 in most places and most years between then and now too.
It's the very nature of the hype cycle that it is very hard to distinguish from a real thing.
--
[1] though that has produced more useful output than some of the previous hype cycles, as I think will the current one as it seemingly already is doing
[2] I was barely born for the start of the “AI winter” following the first such hype cycle
If open source models are good enough (within the category of image generators it looks like many Stable Diffusion clone models are), what's the business case for Stability AI or Midjourney Inc.?
Same for OpenAI and LLMs — even though for now they have the hardware edge and a useful RLHF training set from all the ChatGPT users giving thumbs up/down responses, that's not necessarily enough to make an investor happy.
I personally think the only way AI will end up being a benefit to society is if we end up with unencumbered free and open models that run locally and can be refined locally. Every financial incentive is pushing in the other direction though.
Meta is no doubt doing this because it’s in their best interest, but if both the quality and licensing of LLaMA 2 start a trend that’s a pretty effective counter-weight to eyeball scanner world.
And there’s other stuff. George Hotz is pretty unpopular because he does kind of put the crazy in crazy smart (which I personally find a refreshing change to the safe space for relatively neurotypical people in the land of aspy nerds), but tinygrad is a fundamentally more optimizable design than its predecessors with an explicit technical emphasis on accelerator portability and an implicit idealistic agenda around ruining the whole day of The AI Cartel. And it runs the marquee models. Serious megacorp CEOs seem to be glancing nervously in his direction, which is healthy.
It’s not locked-in yet.
If it isn’t a lot, I encourage you to look more closely. Let’s skip all the disputes about his achievements prior to this year (and you really need to take with a grain of salt anything bad you hear about someone who has been actively fucking up the afternoon of powerful people their entire life).
Tinygrad is so obvious in retrospect, but hindsight is 20/20 and I missed it.
Why the hell do we have kernels for any composite tensor operation? The machine economics are that starting a training run is who cares. Doing a training run is machine hours, days, centuries, millennia.
Spend like a drunken sailor customizing the kernel, exploit the aforementioned “who cares” blank check to just dump the friggin CUDA or PTX or OpenCL or Metal or whatever into /tmp/ and hit it with a fork/execve to nvcc or whatever. You can fuse and columnize and whatever to your hears content in the IR.
We spent how much time rigging up a trouś convolutions via IM2Col by hand with template instantiation in ATen? Well, we’re never getting that back. And while dating NVIDIA is fun, being chained in their soundproof basement not so fun. “It lives with the same 24Gb of GDDR6 on it paid for last year on its skin or it gets the p4d.24xlarge pricing again”.
Nobody has jacked anyone so usuriously via API lock-in since Gates and Win32.
Anyone who is doing a YouTube video called “Get in Losers, We’re building a Chatbot” and then live codes for 5 hours to turn 2700 lines of Python that can’t run LLaMA into 3200 lines of Python that can run it on three times the number of platforms that PyTorch?
I’m struggling here.
Where his arrogance, ignorance and complete lack of experience with how teams and companies function effectively was on full display.
You can spout off technical terms all day long but I've worked at two of the FAANG companies and met hundreds of incredible engineers and it wasn't their grasp of the technology that made them great. It was their ability to lead, deliver, communicate, relate etc which are skills that sound far less impressive but are infinitely more important.
"Hackers like to work for people with high standards. But it's not enough just to be exacting. You have to insist on the right things. Which usually means that you have to be a hacker yourself. I've seen occasional articles about how to manage programmers. Really there should be two articles: one about what to do if you are yourself a programmer, and one about what to do if you're not. And the second could probably be condensed into two words: give up."
I've only worked for one FAANG, but it was for like 7-8 years out of a 20+ year career, and it was mostly on stuff where even small mistakes cost a lot of money so I'll counter your anecdote with one of my own: at the beginning of the century the software business was really weak on demographic diversity (a problem we still have and still needs to be addressed) but it was unparalleled in neurodiversity. In particular there's a longitudinal component to my observation. I'm not here to diagnose anyone via the Internet, but it's pretty clear that George (like a lot of us) is pretty friggin "different".
During the period of time (say 1995-2015 +/- 3 years) when most of the current empires of tech were built, staggeringly successful managers in both open-source (e.g. Linus) and industry (e.g. Sheryl) created a lot of space for aspy nerds who "spouted off" about tech stuff in blunt "this is stupid" kinds of ways, and all kinds of other stuff that's now called "being toxic" rather than "being a weird nerd". It was only during that latter half of this time (and really the end of it) that software became a sufficiently high-status occupation to draw in a bunch of people who decided that it was a good idea to outmaneuver those folks on the org chart and re-brand an entire industry of people from "probably on the spectrum at least a bit" to "toxic asshole".
I don't know if packing Linus off to charm school because he was simply too important to write off was a better or worse move than just letting him be Linus on a mailing list he founded, I kinda liked the original Linus. But even if we decided that old-school hacker aspiness couldn't get a blank check anymore (on the Internet we fucking built), it's just a strictly better idea to strike some sort of compromise with the aspy nerds than to redefine the vibe that had prevailed for decades as basically up there with racism, misogyny, and homophobia (you might notice that aspy tech assholes are on average some of the least racist, misogynistic, and homophobic people in any industry: they on average accept anyone who kicks ass at code or is trying hard and are short with anyone who is dismissive of code, irrespective of demo).
I wrote the `im2col` and dilated convolution kernels on NVIDIA and Intel that went into Caffe2 originally, and AFAIK that stuff migrated over to like ATen or whatever (though I imagine someone has rewritten it since), so I don't exactly love the "spout off technical terms all day log" characterization, but mostly I'll admonish you that calling leadership and communication skills "infinitely" more important than technical expertise is dangerously close to "arrogance and ignorance", and that overlooking that someone has been in the public eye on e.g. Twitter since they were 16 doesn't seem like a particularly serious effort to "relate".
In a way rather reminiscent of early Linus, George has organized a few thousand iconoclastic accelerated computing pros and/or people aspiring to become one into a small but surprisingly credible threat to the current round of Bond Villain monopolists. Maybe we cut him some slack on being a bit of an edgelord on his own Twitter account, his own Twitch stream, his own GitHub Issues page, and to set the culture at his own startup?
History has shown time and time again that you need the guardrails that regulation/laws provides in order to channel progress in an effective direction. Otherwise the negatives of human nature destroy progress. We have seen this recently with the rampant criminality and negative behaviours permanently crippling the crypto space.
In the AI space there is no evidence that allowing models with illegal/unethical content is going to magically translate to some boost in performance, capability or efficacy than would otherwise happen.
In fact again history has shown that the opposite is likely to happen. That having models that don't exhibit negative effects e.g. racism, sexism will result in them being used in more places and exposed to wide audiences. Thus translating to a bigger net benefit for society.
Open is its own area, proprietary general models are a race to zero vs OpenAI and Google who are non-economic actors.
Most AI right now is just features tho, very basic without the real thinking needed.
Next year we go enterprise.
> Make models usable is really valuable Sure... but is there really a moat here? Seems like OSS can address "making models usable" and without lockin from a startup.
Business model is called open core hundred of billions in market cap here as most folk don’t want to build/train their own models and will pay for support and help https://en.m.wikipedia.org/wiki/Open-core_model
Can look at databricks and many kfheds
Edit: if AMD plays their cards right, I'd expect that they can get a lot more in on the action, too.
ChatGPT already has a lot of value by itself, the value added by any startup is going to be marginal at best.
LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.
I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case.
For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
But overall, hasn't that theme been true for like... all tech ever? You have to set up and build your own innovation path at some point.
They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer.
There are, to be fair, many such applications. But it’s not unlimited.
But in general many configurations are possible, but also you need to refine the personas a great deal to get it to work well.
Sure, so you ensemble some results. You're back to the classical "hyperparameter" problem though that's faced ML for a long time-- what those personas are, what those subsequent prompts are, etc. require a fair amount of manual verification and tuning. And the search space is extremely vast.
Not to mention that something like this is likely to be very unperformant.
So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).
I think this is very much like the CCD sensor, that Kodak couldn't envision using because it was "so expensive, slow and low resolution."
This assumes that all AI startups are squarely competing with ChatGPT, or that ChatGPT is some kind of AGI that can do most machine learning tasks making AI startups redundant "thin wrappers" around ChatGPT.
How does ChatGPT make say Weights and Biases irrelevant, or a startup detecting bank transaction fraud, or say a product that detects when someone at your door is a stranger.
If you think these aren't all fundamental units of the next web, you're not thinking about it from the right perspective. If you can't pick apart the real mathematical utility and origin behind crypto efforts from a generation of scammers who hijacked a very real thing, then you just lack understanding or nuance.
We are decades away from the most obvious solution but it very likely involves cryptographically-backed digital currency and smart contract systems used by automated neural networks.
AI benefits from the same economies of scale as all the other means of production, and the winners are going to be the ones that can reinvest their profit into growth and outpace competitors.
tl;dr I don't see distributed multi party computation doing a better job than a rack of H100s
[PS despite my dismissive tone I'm curious if there's something I'm missing]
All cryptocurrency is, is cryptographically-backed digital currency. The current wave of cryptocurrencies are rooted in the blockchain scheme laid out in the whitepaper, with various forms of proof. The only thing new is a way to prevent double-spend; cryptographically-backed currencies were not invented with Bitcoin.
This stuff absolutely has a fundamental place in future commerce, especially as generations grow tired of the payment processor mafia acting as global moral arbiters. Smart contracts build upon this, asset classes such as NFTs give distributed ways to work with authentic data. All of the scammers who jumped on to these technologies have nothing to do with the underlying technologies themselves, nor the core group of people who are still interested in progressing this tech.
AI is just automation, which can make use of these tools in a trustless environment, without disruption from untrusted/unwelcome parties. It's not life-changing stuff, but these tools will fundamentally drive the web in ways you won't even notice if you don't look for it.
At the end of the day, the value add is also around integration and implementation and that is very difficult to generalize.
Edit: fixed the number - I thought it launched in January. It turns out late March was the launch, while the first hints/discussion about it were January. I got around to it in early July.
I’m not sure what you mean by this; GPT-4 launched 4 months ago.
For example, it's relatively straightforward to generate dialogue or other text for a game, but structurally connecting any of that text to game mechanics is unsolved (though AI Roguelite is trying).
Many users' GPU's are powerful enough to run models locally, but the rub there is that you can't really run the model at the same time you're playing the game normally, unless you want, like, massive frameskips.
Producthunt has basically become that these days, none of it is inspirational nor value adding, just constant "X but with AI"
That said, you don't have to be the mega players to have an existing small moat. If your product does something great already, you get to improve it and add value for users very quickly. That's been my experience anyway.
This is assuming your thing is one call to GPT-n rather than a complex app with many LLM-core functions, and it also assumes that data is easy to get.
Chatbots are going to become "features" in more tangible products, not "products" themselves that people are going to buy (or at least, they sure aren't going to buy them from "value add" resellers).
We've barely scratched the surface of what generative AI can do from a product perspective, but there's a mad dash to build "chatbots for $x vertical" and investors should be a little skeptical.
By just replicating a ChatGPT interface but for Your Taxes (TM) it's really a huge slap in the face to computer users that already can't tolerate typing data in.
It's just software, there's little "secret sauce" in the engineering, it's the knowledge of the customer problem that's the differentiator.
That is, a lot of people are thinking at the level of "let's build a model" but for a business you will need to build a model and then update it repeatedly with new data as the world changes and your requirement changes.
There would be a lot to say for a solution that includes tools for managing training sets, foundation models, training and evaluation, packages stuff up for inference in a repeatable way, etc.
One trouble though is that you have to make about 20 decisions or so about how you do those things and developing that kind of framework people get some of them wrong and it will drive you crazy because other people will make different wrong decisions than you will. (To take an example, look at the model selection tools in scikit-learn and huggingface. Both of these are pretty good for certain things but they don't work together and both have serious flaws... And don't get me started with all the people who are hung up on F1 when they really should be using AUC...)
So given the choice of (a) building out something half baked vs (b) fighting with various deficiencies in a packaged system, you can't blame people for picking (a) and "Just doing it". (Funny enough I always told people at that startup that we'd get bought by one of our customers, I thought it was going to be a big four accounting firm, a big telecom, or an international aerospace firm but... it turned out to be a famous shoe and clothing brand.)
Contrast that how quickly Adobe rolled out Generative Fill, a product that will keep people subscribed to Photoshop. (e.g. it changed my photography practice in that now I can quickly remove power lines, draw an extra row of bricks, etc. I don't do "AI art" but I now have a buddy that helps retouch photos while keeping it real)
If they went and screwed around with some startup they'd add six months to a project like that unless it was absolutely in the place where they needed to be.
(2) If you were like Pinecone and working on this stuff before it was cool you might be a somebody but if you just got into A.I. because it was hot, or if you pivoted from "blockchain" or if you've ever said both of those things in one sentence I am sorry but you are a nobody, you are somebody behind the curve not ahead of the curve.
(3) I've worked for startups and done business development in this area years before it was cool and I can say it is tough.
Somebody who needs a system built for their business right now gains very little talking to them.
If a startup is a year or two post funding it might really have something to offer, but the huge crop of A.I. startups funded in the last six months have missed the bus.
Big co's can frequently move very fast when there is a lot on the line.
Just like forms over SQL, there seems to be a never ending demand.
If one can scrape the data from the web, I can't imagine having much of a moat or selling point.
2. data. Can't do anything custom without good training data! How to get this varies widely across industry. Partnerships with established non-tech companies are a common path, which tend to rely on the network and background of founders.
Even with both those things it's not easy to outcompete a large, motivated company in the same space, like a FAANG. They have the researchers, they have the data and partnerships, so the way to beat them is to move quickly and hope their A- and B-teams are working on something else.
They were L7/L6 ML researchers/eng at FAANG, I'd bet there are quite a few people like that lurking here.
The Mistral folks have impeccable timing, but are leaving FAANG somewhat late compared to their peers.
If you know how to run a Python script, you can fine-tune a LLama model:
As an example, the startup-employed AI researchers I know had already PEFT'd llama2 within a day or two of the weights being out, determined that wasn't good enough for their needs, and began a deeper fine tuning effort. That's not something I can do, nor can most people, and it's a serious competitive advantage for those who can. It's a rather different interpretation of "can adequately fine-tune" than "can follow a tutorial".
When I think "AI startup", I think of the places where these people work. I don't think there's many of those people, and I think their presence is a big competitive advantage for their employers.
As you can see with all the responses here, they have failed to realize that this is a trick question.
The real answer is that none are special and can be replicated by tons of competitors.
So all these AI startup companies depending on cloud AI services or even open source models have no moat.
Only the same big tech incumbents.
All the usual things.
First mover
Features
Integrations
Platform synergies
I've also heard some companies that build the LLMs say that those LLMs are their moat, the time, money, and research that goes into them is high
Other than that, I haven't seen anything with monopoly potential coming from startups so far. This is what VC is looking for, though: The potential to make a platform to rent out to people who produce the value, curate it and pay for it. Basically, the next "academic publishing".
2030, day 8 of the 17th AI boom: A starry-eyed founder shows up to a VC office with a pitch-deck for their GPT-47-based startup which automatically responds to Yelp reviews, only to be turned away; the VCs are done with that now, and will be doing robot dogs for the next week.
Also,
> Unlike during the dot-com bubble of the 2000s, AI isn’t entirely based on speculation.
I'd say the dot-com bubble was backed by a revolutionary product: the Internet. That doesn't change that expectations were too high.
Some of the companies involved are now worth trillions.
I suppose you could argue Google, but it's an odd one; it was right at the tail end, and was really only taking off as everything else was collapsing
Also, Netflix was founded Sep 1997. Also, Paypal, founded Oct 1999. Also, Ebay, founded 1995, [del: though Ebay and Paypal together today are worth only 23.8 billion dollars. :del]
I think a lot of the money made by founders and angel investors during the dot-com era was made by selling a startup to an established company for 100s of millions of dollars, which is pretty good money since it often took only took 5 years to go from the founding event to such an exit, but I cannot think of an easy way to get a list of such exits.
ADDED. In writing the above, I assumed Paypal was still part of Ebay. In reality they are separate companies again with market caps of 23.8 and 84.59 billion dollars.
Interestingly, Netflix has its origins in a... more typical dot-com-era story: https://en.wikipedia.org/wiki/Pure_Software
They were just too early I reckon. I wonder what we're "too early" for today.
Software velocity is increasing. Investors should be considering what that means for their investments.
I would be worried if I were tied up in a company that depends on bloated professional services. LLM-enabled senior engineers are 100X more efficient and safe than brand new junior devs. These organizations that embrace the best people using the best tech ought to make Oracle and their famous billion dollar cost overruns quake in their boots.
Come on man. Having seen the inside of big-tech-TM and the senior engineers there, yes they are fast and good, but they are not 100x better than the new guy. Maybe 3--5x at best.
Anyway how do you train good senior engineers? They don't just pop up out of thin air.
Even an LLM enabled senior engineer was 100x better, the situation has to be the right one where all 100x can be applied productively.
An analogy -- if I am an athlete who runs a mile 100x faster than my competition, does that mean that my country will win 100x more medals at the Olympics? Not unless I compete in every competition.
Now, if you can make a system that consistently creates or recruits (or even just prevents your competition from finding) more productive developers you can get a real advantage. And, as soon as you do it you will have people imitating your strategy and tactics and poaching the people who are most important to building the system. It is way more complex than providing an LLM.
More like infinity since juniors usually can't independently progress on non trivial tasks (and will probably regres if left to try).
I sort of get what OP is saying, if LLMs get better you can delegate stuff you usually delegate to juniors.
But that's nowhere near reality right now, LLMs are not even a net positive with all the failure modes and poor workflow.
It takes too much compute to get meaningful context. Having to summarize and chunk problems is more work than the benefits.
Copilot is great at speeding up obvious typing, but that's like 10% bump.
Don't underestimate the industry's capacity to self-sabotage on the long-term for a short-term pay-off. This happened with de-industrialization in the West and might happen with Tech. You only need these senior developers until you realize that you haven't trained any new senior devs. But that problem is for 20-years-in-the-future CEO. No one cares now as long as you can keep next quarter revenue/profit up.
So what substantive and defensible advantage is your money buying in the AI ethos when this effect is essentially inevitable?
Answer: not much.
So it’s very logical that the team, book of business and the tech platform itself are what are driving valuations.
During the PC revolution you could buy apple hp and Microsoft and know that you were capturing the hardware market. Here we see Nvidia, AMD, Apple, and Microsoft (somewhat) looking like the major beneficiaries and the market is following that. Maybe it becomes a Omni-platform market and people rush into OpenAI once public.
you can start by googling the above, for example
The type of solutions I am sorely missing have to do with adding data/work. E.g.: don't tell me what code to use, find me stackoverflow entries that might be highly relevant instead! Don't tell me what the data I am looking at means and have me google that separately, use LLM contextual "undersranding" to find best source material describing what I am looking at or helping me piece together a bigger picture!
Because that seems to be the path we're actually on. I suppose there's plenty for investors to love in that equation, but it's a little harder to attach a hype machine to that.
We are following normal trajectories here - we are in the skeuromorphic phase where we adapt things form one system to another - X, but with AI. Think early iphone - tape recorder , notebook, compass. Those phases tend to benefit agile incumbents with tech companies generally are.
Next comes the phase where we make use of the technology in a native way, new products, etc.
That’s where you ought to be investing.
The second issue is that the user interface is captured by rent seeking incumbents. Many investors want to be that and are disappointed by anything that doesn’t open that opportunity. Crypto had it for them because you’d cut out the bank incumbents. AI is gonna live on platforms we already choose, so not as exciting.
They are also realizing that the many of these new 'AI startups' using ChatGPT or a similar AI service as their 'product' are a prompt away from being copied or duplicated.
The moat is quickly getting evaporated by $0 free AI models. All that needs to happen is for these models to be shrunken down and be better than the previous generation whilst still being available for free.
Whoever owns a model close to that is winning or has already won the race to zero.