At this point it is unstoppable and $0 free local AI models will only accelerate and OpenAI and Google knows it.
Sun, DEC, Microsoft, etc just assumed they would use their cash and market presence to dominate the web up, down, left, and right.
LAMP showed up and ate their lunch. Today open source is the underpinnings for basically every startup over the past 25 years and everything from Android to MacOS to every browser rendering engine. An exclusionary list would be easier.
A 2008 study[0] from the Linux Foundation estimated the development cost of Fedora (at the time) to be almost $11B - in 2008 dollars on a 15 year old codebase. Wonder what it is now?
Point is, OpenAI raising a few billion dollars sounds like a lot of money until you look at the aggregate collective.
FOSS has always had commercial backers, entities with interesting business plans, non-obvious strategy (at the time), etc.
It’s perfectly clear to me why Meta is doing this. It’s also damn near breathtaking to watch the FOSS communities build, build, and build on top of the work of those like Meta as well as each other.
Personally I don’t understand how anyone could think OpenAI, etc vs FOSS is going to be special or unique against what we know to be obvious. I don’t see how this could possibly go any different than the early web. Why, just because it’s “AI” instead of “WWW” this time?
OpenAI will end up like Windows - used by some of the masses and corporate/Enterprise applications, users, etc (think Office integration). Everything else will be a herd (yes, Llama/Alpaca/Vicuna/etc reference.
[0] - https://www.linuxfoundation.org/press/press-release/linux-fo...
APIs... it's all about APIs and ease of use. Companies have data centers or could have data centers if they wanted, but a lot use AWS, MSFT, GCP, and friends.
Let’s say I’m a law firm or a medical group. Do I need ChatGPT to write me a bedtime story or tell me about the Great Chicago Fire, or write code?
No, I need it to help me write my briefs, summarize patient notes, whatever.
The next crop of startups will be (and already are) LegalGPT and ScribeGPT (or whatever) with fine tuned highly specific, highly curated, regulatory compliant, quality controlled, decision controlled, PHI-scrubbed, tuned on your own data, etc, AI solutions largely based on open foundational models, open serving frameworks, data pipelines, etc. Available via end-user interface and/or API.
Then watch LegalGPT be acquired by LegalZoom, LexisNexus, whoever. ScribeGPT by Epic, etc. or vice-versa.
ChatGPT is jack of all trades, truly master of none. As I said it will always have its place but it will recede more and more.
AI is trying to catch up to human intelligence. Do you go to a mechanic when you should be going to a dentist?
The difference is after 15 years the entire crypto ecosystem has somewhere in the range of 50-100m "users" worldwide (at best). ChatGPT alone has an estimated 100m monthly active users after less than a year...
You're correct in making the comparison but user adoption is the best metric to separate hype vs utility and AI is already absolutely blowing past crypto (obviously).
The output will be significantly slower than what you’d get from GPT-3.5, so if your time is worth anything then that will be a huge factor that’s hard to calculate. Just assume everything will take about 3 times longer though.
If you amortize the cost of the 3090 for as long as you’d expect it to be relevant, you can estimate a rough monthly cost of the hardware. I’d give it just over a year at the current pace before you’d probably want a replacement. That’s about $50/mo.
Operation at commodity scale with the still significant lead OAI or Anthropic has on efficient operations has its advantages beyond just SOTA performance.
Obviously we all know now that it was well worth it, this is how we got our start, and likely why we’re on HN today.
That RTX 3090 can also run thousands upon thousands of other models for any number of use cases at the same cap and op ex. It can enable and build anything from a sense of wonder to an extremely valuable, fulfilling, and long lasting career. Worst case it’s fun, a relatively cheap hobby. Speaking of fun it also plays games.
The RTX 3090 is an investment, not a cost.
That said, some fair points generally but:
1) 25 hours of use is A LOT. GPUs scale power consumption too. The GPU will use slightly higher than idle power to keep the model in memory, then spike to whatever your power limit is while the model is executing. Let’s say 30 seconds to generate as worst case. Human reads that, comprehends it, does something. Meanwhile the power scales back down until you hit it again. Point is, 25 hours is 3,000 30 second generation sessions - 100/day, every day. For $1 (plus base/idle load of machine). You can also suspend the machine or turn it off to cut down on idle-idle waste.
2) Are you doing any optimization? I mostly work on high scale and inference serving frameworks (not on 3090s) but if I had to guess with something like lmdeploy, AWQ int4, int8 kv cache, etc you should be far exceeding ChatGPT in tokens/s. For example, I do concurrency testing with 13b on the RTX 4090 in my workstation. It hits 115 tokens/s - beating the 15 tokens/s or so from ChatGPT should be pretty straightforward with 34b on an RTX 3090.