329 karma · joined August 5, 2023
[1]: https://news.ycombinator.com/item?id=49322280
That's not what the GP said, they were referring to someone who "builds their own moat where they solely can profit and excludes others from the freedoms they themselves enjoyed".
I don't want to speak for them but I guess that LLM companies which release exclusively open weights models, or better yet, open weights plus training pipeline sources, would not fall into this category.
And these will be gone, too, a couple months later. Or possibly even at the same time.
Yes
>and do tools like Github and Confluence render them?
No
The preprint is interesting in that it predicts that on the way there, one can run into some kind of vicious cycle that leads to suboptimal outcomes even for the company owners. Navigating around this would certainly slow this process down, but I personally don't see it changing the ultimate outcome.
Where they went wrong, in my opinion, is in the implementation details.
It's mostly death by a thousand cuts: Requiring reputation to gain the ability to post comments, then having one's answers deleted as "this should've been a comment". Overeager marking of questions as duplicates, e.g. despite the equivalence between two situations being non-obvious (e.g. someone asks about data type A, and it turns out that it's a subtype of B for which an answer that applies to both exists; that should not be a duplicate, the fact that it's a subtype is the answer!). Endless other decisions like that, which wouldn't have taken any extra effort to implement correctly.
One feature they could've built that would have taken effort but also greatly helped against the common newbie complaint of "hostility" would've been a "newcomer track", which would've been more forum-like and guided them towards either formulating a good question or seeing that's it's already answered. In the latter case, some of the keywords that came up during this process should've been fed back into SEO so that future newbies would become more likely find the answer via a search engine despite using clumsy terms. I think they tried a simpler and worse version of this idea towards the end with "staging ground" but by then it was too late.
While this seems to be allowed because the current ToS don't seem to explicitly forbid it, I'd be surprised if this loophole stayed open for long... Why would they even distinguish between business and (much cheaper) individual plans if companies can work around it by telling employees to just pay for the latter themselves?
This one stood out to me:
>Machine-scale infrastructure. [...] Git itself wasn't designed for that load, and bolting AI onto platforms not built for agents is the biggest mistake of this era. [...] Git itself is being reengineered for machine scale.
Git itself is so far down the list of bottlenecks that do or could hamper LLM-driven development, even projecting years into the future...
On the other hand, LLMs seem perfect for triage and finding duplicates, so it's still surprising that they've let it get this bad.
Use Mypy in strict mode and run it in the post-turn hook of your LLM harness so the LLM has no choice but to obey it. And don't use overly general dictionary types when the keys are known at development time; use TypedDicts for annotations if you must use dicts at runtime.
You could do both experiments with dogs instead of humans and roughly 100% of dogs wouldn't manage to shoot themselves with the gun, whereas if you forced them to press one of the two buttons (e.g. keeping them in a room until they press one by chance), roughly 50% would press the red one. So the two experiments differ strongly w/r/t to how likely it is for a "non-thinking" organism to choose each option.
I don't remember who, but someone made an interesting point about this around the time GPT-4 was released: If the major nuclear powers all understand this, doesn't that make nuclear war more likely the closer any of them get to AGI/ASI? After all, if the other side getting there first guarantees the complete and total defeat of one's own side, a leader may conclude that they don't have anything to lose anymore and launch a nuclear first strike. There are a few arguments for why this would be irrational (e.g. total defeat may, in expectation, be less bad than mutual genocide), but I think it's worth keeping in mind as a possibility.
E.g. it is mentioned that MMAcevedo performs better when told certain lies, predicting the "please help me write this, I have no fingers and can't do it myself" kinda system prompts people sometimes used in the GPT-4 days to squeeze a bit more performance out of the LLM.
The point about MMAcevedo's performance degrading the longer it has been booted up (due to exhaustion), mirroring LLMs getting "stupider" and making more mistakes the closer one gets to their context window limit.
And of course MMAcevedo's "base" model becoming less and less useful as the years go by and the world around it changes while it remains static, exactly analogous to LLMs being much worse at writing code that involves libraries which didn't yet exist when they were trained.