Please don't release new models unless there is an arch change. All these new LLMs are essentially the same technique applied to almost the same dataset.
No one is making you pick up a shovel to build alongside them; instead you choose to rest on your laurels and complain about other peoples' hard work and dedication to providing people with choices.
[0]: Android uses the Linux kernel which is almost the same across distros, but isn't per se a Linux OS. I'm talking about real Linux running on a mobile phone.
Have you ever considered that these people are satisfied with their interests and truly could not care less about your opinion? Or that your opinion is just that-- yours? Not some absolute truth?
Anyway, it's beside the point, as there are multiple high quality Linux distributions to choose from, thanks to a large de-duplication of efforts through libraries.
How about GQA vs MHA, or GQA vs MLA?
If anything attention-like is same in your mind, is S5 and RWKV different arch given that both are some kind of linear RNN?
The closest is mixtral 8x7b but that one only uses a fraction of its parameters for each pass. This one should produce better but slower results at roughly the same memory requirement.
One nice thing is that all three of these models are Apache 2.0 licensed.
Let these researchers do what they want, they didn't release it for you specifically.
You simply don't know what you're talking about. Your overly cynical take is against Hacker News guidelines.
You have added nothing substantial to this conversation. If you don't have anything substantial to say, then you should stop attempting to simply instigate. Please review the HN guidelines. https://news.ycombinator.com/newsguidelines.html
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
Have a good day, and please reconsider how you interact with others on this website before telling others to do the same.
I might be feeding the troll here, but building a new LLM architecture is as far from science than building a new bridge architecture is. That is, it might use some jargon and apply some concepts, but it is not scientific. Talking about gaslighting is uncalled for.
> Most scientific research is funded with the hopes of seeing a return of investment.
It is irrelevant, as you so helpfully put yourself. And also, wrong. Most scientific research is funded in the hope of seing applications. Return on investment is at best a secondary objective. You cannot run a research lab and expect to break even in monetary terms.
> You simply don't know what you're talking about. Your overly cynical take is against Hacker News guidelines.
But then, so is your overly aggressive take. So can we please stop for a moment and have a constructive discussion instead of calling each other names?
HN guidelines suggest to take each comment in good faith.
> building a new LLM architecture is as far from science than building a new bridge architecture is. That is, it might use some jargon and apply some concepts, but it is not scientific.
If you would like to make the case as to why, I'm all ears, but simply stating such without stating why is hardly a substantial argument.
> Talking about gaslighting is uncalled for.
You're right, gaslighting is a strong accusation. I should have just pointed out the gatekeeping and left it at that.
> Most scientific research is funded in the hope of seing applications. Return on investment is at best a secondary objective
Again, I should have proofread my comment better and used a positive form such as "much of" or "a lot of", instead of a comparative or superlative form. I didn't intend to make any claims as to the exact ratio of research funded with financial motivation.
> But then, so is your overly aggressive take. So can we please stop for a moment and have a constructive discussion instead of calling each other names?
That's where I disagree. I didn't call OP any names, I do not intentionally engage in ad hominem. Describing behavior is not calling someone names. I'm all for a constructive discussion, and I will consider your critique on my admittedly hastily written comment, but you also need to find it within yourself to take a more charitable and constructive approach, and to not make unsubstantiated claims.