1,402 karma · joined December 16, 2013
I hope they release a good root cause analysis report.
OpenAI is receiving far more data with a far greater privacy impact than social networks. And all this is happening at a time when the US is transitioning from a somewhat functioning democracy to an autocratic and fascist system.
https://www.rollingstone.com/culture/culture-news/chatgpt-pa...
What's more important is trust. What's to stop them from changing the licence again after evaluating the impact of the first change? Liquibase has collected "anonymous" metrics by default since version 4.30 (do they consider an IP address and timestamp as PII?). As soon as I saw that, I anticipated this scenario. It was not really a surprise. They have a way to analyse the impact. Now, I am reassessing the risks.
I have nothing against someone trying to monetise useful software. However, switching from an open-source software (OSS) licence is essentially a bait-and-switch tactic. This immediately destroys trust. It also destroys the part of the user base that is difficult to monetise but still has the potential to be monetised. I was hoping that the Elastic and TerraForm debacles had taught people a lesson.
Flyway is also questionable at this point. If Liquibase is switching, what's to stop Flyway?
Unless a fork is happening, I'm considering creating my own migration library tailored to our actual needs and usage. It should not be so hard. Liquibase was more of a convenience.
Even minor changes to models can render previous prompts useless or invalidate assumptions for new prompts.
> I wanted to create an architecture that minimized future work on both the admin app and the controller network layer. That way, the business could lean on its in-house development experience to make changes after the project.
From your perspective as a UI developer, you would need to update all clients (in the building) if something on the server changes, e.g. if a new button or game state is added. HATEOAS is a good architectural choice if you want to minimise the likelihood of having to change the client implementation.
I use HATEOAS as much as possible in my daily work. It is a good way to think about state, state transitions and what capabilities a client must have. That's the actual part of the author's motivation. It provides the "in-house development" with paradigms how to make changes.
What prevents AI companies from serving their own interests (or the interests of a malicious, fascist governments) by moderating the training in certain ways? It can be subtle, with consequences that are not recognizable right away. Didn't Musk already complained about Grok being "too woke"?
And how can I trust those companies with my own data?
P/E ratios - VW 6 - BYD 13 - Tesla 200
Tesla's valuation is absurdly crazy.
Yes, you can use Genie 3 to train other models. Its far from perfect. You still need to train Genie 3. And its training and outputs must be useful in the context of what you want to train other models with. That's a paradox. The feedback loop needs to produce useful results. And Genie 3 can still hallucinate or produce implausible responses. Symbolism is a wide term. But a "World Model" needs it to make sense between concepts (e.g. Ontologies or the relation of movement and gravity).
This is the most important sentence describing the fundamental issue that LLMs have. This severely limits the technology's useful applications. Yet OpenAI and others constantly lie about it.
The article very clearly explains why models won't be able to generalise unless RL is performed constantly. But that's not scalable, has other problems in itself. For example, it still runs into paradoxes where the training mechanism has to know the answer in order to formulate the question. (This is precisely where the concept of World Models comes in or why symbolism becomes more important.)
LLMs perform well in highly specialised scenarios with a well-defined and well-known problem space. It's probably possible to increase accuracy and correctness by using lots of interconnected models that can perform RL with each other. Again, this raises questions of scale and feasibility. But I think our brains (together with the other organs) work this way.
All these things are not black boxes and they are mostly deterministic. Based on the inputs, you EXACTLY know what to expect as output.
That's not the case with LLMs, how they are trained and how they work internally.
We certainly get a better understanding on how to adjust the inputs so we get a desired output. But that's far from guaranteed at the same level as the examples you mentioned.
That's a fundamental problem with LLMs. And you can see that in how industry actors are building solutions around that problem. Reasoning (chain-of-thought) is basically a band-aid to narrow a decision tree, because the LLM does not really "reason" about anything. And the results only get better with more training data. We literally have to brute-force useful results by throwing more compute and memory at the problem (and destroying the environment and climate by doing so).
The stagnation of recent model releases does not look good for this technology.