Hacker News Reader

Comment by pksebben | Hacker News Reader

Glad to see more progress on open(ish) source versions. There's so much more these things could do unfettered by corporate motivations.

View on HN

I'm convinced this is going to be history repeating itself:

- Microsoft/Sun/etc trying to own web in the late 90s - early 20s. LAMP came and ate their lunch (for all intents and purposes).

- Microsoft and Windows Phone. Android (open source again) plus Apple but with the BSD/Mach underpinnings could be argued.

- Microsoft Edge. Give up, use Chromium.

Once again we have Microsoft (famously via OpenAI) doing what they do and trying to own an emerging space. Based on the lightning progress in the open(ish) "AI" space I'm pretty certain OpenAI and others will take a back seat to the open ecosystem within a few years.

According to interviews OpenAI only released ChatGPT in advance of GPT4 because of paranoia that they would be supplanted by open versions and end up being irrelevant. Their fear is not unfounded as it-just-happened to them, with Dalle-2 and stablediffusion.

OpenAI is well funded but again I'm reminded of open source. Way back when in 2008 the Linux Foundation (yes, consider the source) estimated[0] that Fedora 9 represented approximately $10.8B (2008 dollars) in cost if developed commercially/conventionally. I actually believe that (Debian as another example has over 50k packages). Meanwhile it has to be some multiple of that 15 years later.

OpenAI having a few billion or more to throw around seems like a lot. The combined rest of the world including supporting commercial entities (Stability AI and others = Red Hat, IBM, Intel, FB, Google, etc) and open source contributors have the equivalent of many times that.

On a long enough timeline the closed/proprietary approach cannot win.

[0] - https://www.linuxfoundation.org/press/press-release/linux-fo...

> On a long enough timeline the closed/proprietary approach cannot win.

Yes it can. The magic word is "regulation".

We are aligned. I just wonder when we'll get to a stage where we can start producing value faster than the landlord types can charge rent on it.

I think this tech might actually represent an opportunity to break out of the systemic quagmire we've been in societally for so long - rotting institutions dictating access to information and entrenched powers accumulating for the sake of accumulation.

I want everyone to have a personal assistant who can help them learn whatever it is they want to learn, for free, at any time of day. We're so damn close.

Maybe. The problem is that you need billions to train new models.[1] At least with how things are now.

[1] https://techcrunch.com/2023/04/06/anthropics-5b-4-year-plan-...

With 6-7 more doublings in compute power per watt, consumers will have the power of 1000 A100 GPU's in their iPhone. "Eventually" will come before I die, at least. That would happen outside of GP's time estimate but a moderately funded university consortium could probably afford it just a few doublings from now.

I think we'll reach architectural changes that make this moot before we reach hardware for it. The way we train these models is constantly in flux, and we just need someone to crack continuous learning so we can pass models around and train them en-masse, using the collective unused compute that is literally sitting on mine and everyone else's desk right now.

When considering future of tech, its valuable to consider there are at least two well-trodden paths:

1. semi-linear extrapolation of existing tech and progression (maturing tech)

2. new paradigms approaching the problem from a new angle or with new insight that invalidates or levels up past 1.

Since we're in the midst of a cambrian explosion for both 1. and 2. IMO I dont expect limitations as we've been seeing them will hold up even under the medium term.

Reply on news.ycombinator.com