Comment by rektide | Hacker News Reader

Comment by rektide | Hacker News Reader

The whole Enlightenment idea that begat modernity was that man could learn from the universe. We could use science to learn, improve, grow & progress. And that is what we expanded the future with, jointly, with Royal Academies of Science & newly begat intellectual peerships.

AI defies these roots we have sprung from. Mankind will not have access to this artificially created universe. The future now happening is safeguarded deep inside corporate keeps. Answers will be shared but understanding will be withheld. We will not be able to study or learn about this universe.

When we say this is The Dark Timeline, it's very much because the light from this era will not expand. The progress made here is not shared, is not free to spread, and will inevitably be lost, likely replaced with newer also corporate controlled automated-answering-machines. This is an entirely asymmetric future, a springboard into the dark, replacing humans and human knowledge with insubstantial easily manipulated highly controlled unobservable non-participating un-democratic artifice. If you care about humanity & the human cause, you should fear what's happening here & detest it.

it sounds like your concept of the fear isn't AI comes and destroys us all like a lot of people fear, but rather huge corporate walls only increasing the disparity between the weak and the powerful, hoarding this stuff to themselves. I agree with this take and to me that is the most realistic. We desperately need OSS LLMs and democratic access to all of this goodness that AI will bring. I dont want black box subscriptions

My fear is that they actually succeed in creating AGI and us plebes are no longer required. Do people actually think that AGI will herald a new era for humanity like in Star Trek? No, the wealthy and powerful will just eliminate the worker class.

its definitely the darker timeline. i do think its unrealistic unless there are certain ways to protect themselves from uprisings/revolutions. unless they build a castle in the sky hovering in LEO or AGI builds sentries, that will be unlikely. i imagine the more likely scenario is systemic injustices that aren't as obvious but carefully controlled to keep the workers "happy enough." kind of similar to today's age, except there is some tiny percentage of chance to become rich, whereas in the future it will be literally zero with a thin veil of hope

To add to your point, surveillance is a key component in such a system, since "undesireble" behavior can quickly be detected and punished. The current zero-privacy interfaces are a huge step in the wrong direction.

This is my fan theory of The Terminator: that the machines never took over. They worked for billionaires and the billionaires wanted the rest of humanity to THINK the machines had taken over.

It's unlikely that OSS LLMs will ever be able to compete with corporate LLMs. I can only think of a few scenarios where this could work:

1. Someone develops a procedure for training models with distributed computing resources, including consumer CPUs and GPUs. Even this is not really guaranteed to work, since corporations will probably just buy up all the consumer GPUs, since they are cheaper on a FLOPS/$ metric.

2. One or more governments provide a lot of funding for OSS models, including being willing to pay competitive salaries for the best talent (potentially millions of dollars per year). This is unlikely for a lot of reasons. The only thing that could speed up the process enough to compete with private organizations is a major war that required AI to win. In that case, though, open source would be the least of their concern.

3. Scaling laws stop working and Moore's law catches up. In that case adding more compute won't really help and eventually even organizations with small budgets can afford to train a SOTA LLM. We can't know until we find the limit, but we haven't hit the limit of scaling laws so far, despite scaling up massively over the last few years, so I doubt we will find the limit any time soon.

4. A bunch of corporations that have no hope of reaching first place decide to combine their resources to beat OpenAI and thus prevent a monopoly. I'm not sure if there is precedent, but even if there is, that would still require a lot of coordination and resources for little direct monetary gain.

We'll see what happens, but I'm not really confident about any of these possibilities.

I think this is inherent to the technology itself. It's a black box the owners can tweak but the users cannot and the users are expected to trust it at face value.

It's the idea of going to a corporate entity's black box in order to know how to live. It's a blind trust that is truly blind.

This doom and gloom prediction paints past scientific achievements with a rosy patina that is unjustified. Major scientific endeavours have always required funding from less than palatable sources, and while understanding may initially be withheld behind corporate moats, it cannot be contained there forever.

Previously the world was material & it was extremely hard to make something good & useful & leverage it well while denying access to it, while keeping it secret & unobservable.

But this virtual construct does not actually live in the real world. No one can poke or prod the systems, no one can see the models it was trained on.

These are dark by default, are so immaterial as to be unlike anything else mankind has experienced. And their impact & use, how they change the world is almost unlimited; global connectivity gives them unlimited access & breadth, from within the iron mountain security of their corporate keeps.

Even if the superhuman corporate bodies did decide they want to share, theres such unusual barriers to discocery & learning here. Assuming the petabytes of training models are made available, who is there that can accurately review them? Who can train their own systems? How can we experiment with other trainings, other systems? This is a megaproject, one that produces unique distinct gems. Watson's quip come back again, in a different form, "I think there is a world market that can maybe afford five AI."

The ratio of how leveragable it is versus what the world can see or ape has gone quickly towards infinity. Nothing like this exists, this is unprecedented entirely.

Oh I don't buy this "dark and immaterial" argument. There are many physical processes that seem magical to those unaware, and having access to the final product may reveal nothing about the processes used to make them.

As an example consider gunpowder. Having the finished product doesn't provide any insights into how it was made. It requires, what is now basic, but what was once, a pretty strong grasp of chemistry.

Another example is antibiotics. These life-saving drugs are almost exclusively created by for-profit corporations, yet despite over a hundred years since their invention, they remain easily accessible and some would argue too plentiful.

> Nothing like this exists, this is unprecedented entirely.

The above examples illustrate that life-changing inventions are par for course in human history and represent the cornerstones of the betterment of our species.

The early groups that had gunpowder used that power to inflict a lot of harm on groups who didn't. Same with many other technologies, and I expect AI will be the same. Except that unlike gunpowder, which was often controlled at the nation-state level, successful AI will largely be controlled by a handful of billionaires.

Exactly my point. I was trying to illustrate for the OP's benefit that this is not the first time in human history that game-changing tech was invented.

Gunpowder was controlled by kings for the longest time. No different from modern billionaires or oligarchs, heck monarchs had greater control and inflicted extensive misery. But despite all that, despite the control, the misery and the harm, do you seriously consider that we should not have invented gunpowder?

Gunpowder is an interesting case to suggest. It was already well known around 0AD that saltpeter emitted a dangerous poisonous smoke, already was clearly an interesting element anyone could go mine & put to use hurting people with. It's already overtly a mineral that, like, doesn't conflagrate, but does pretty weird interesting shit when on fire, and lots of folks figured that out.

It seems like by the time 13th century medeival Christiandom rolled around there were two main sources, Aragon & Montpellier. The limited supply indeed did probably greatly slow down the understanding & use.

And indeed, you can make saltpeter... by collecting & refining excrement. Queen Elizabeth paid 300 pounds of gold to the Germans after going to war with Spain & thereby losing access to Spain's natural extract.

The Chinese developing actual black powder is another interesting case. Known for centuries, it wasn't until 904 we had the first recorded use of it for combat. By the 11th century the Song dynasty started greatly controlling access & trade.

In general I think there's probably a ton of primitive chemistry/discover even much earlier ages could have done with access. The actual material being somewhat not easy to scare up is definitely an issue here.

I think the situations you're talking about don't apply very well to the moderm world. The Enlightenment project of building instruments & understanding natural science have built up, and today it would not be hard to figure out how to reproduce black powder. There is definitely more murkiness still with biology, with figuring out how to concot some chemical compounds, or even in material science with figuring out what conditions a certain combination of elements were brought together in. But these historical challenges rely primarily on a darker time in human history to contain the not-so-secret. Enlightenment observation won, it brought us out of that age, it gave us tools to figure things out with.

Im interested in being challenged more, but holy heck AI still feels enormously forbodingly dark & immaterial. Yes, bring able to observe a thing counts. Trying to persuade me that having zero access is OK or normal in the world doesn't sit with me well at all.

These models aren't tools that are built once and then released to be used freely. They require immense resources to stay at the cutting edge.

This is a very different scenario than something like the first PCs. Early computers could be taken apart, experimented with, and explored freely. This allowed countless creative uses to emerge and evolve.

The headline LLMs are the exact opposite of that. Every single use is checked and anything the owner dislikes is blocked. Many types of exploration are actively discouraged. Usage requires continuous payment and connectivity.

> The future now happening is safeguarded deep inside corporate keeps.

Is it though? It seems to me the current AI revolution is much as corporate enterprise as it is hackers running models on their own computers. In that sense, it's no different from any other technology.

Reply on news.ycombinator.com