HNHacker News
TopNewBestAskShowJobs

suziemanul

41 karma · joined July 27, 2023

submissionscomments
suziemanul··on [dead]
We're hosting a boxing-style debate in San Francisco on May 5th between Łukasz Kaiser, co-author of "Attention Is All You Need", and researchers building post-transformer models.

I'm the CEO of Pathway and co-author of the Dragon Hatchling (BDH), our post-transformer architecture. We're organizing this to bring key researchers on both sides to debate each other without any BS. We invited Łukasz specifically because he can push back on the post-transformer thesis harder than almost anyone.

Confirmed so far:

- Łukasz Kaiser, co-created Transformer, ChatGPT, o1, o3, TensorFlow - Mathias Lechner, Co-founder/CTO of Liquid AI, co-created liquid neural networks (MIT CSAIL) - Adrian Kosowski, CSO of Pathway, created BDH (ex Inria, École Polytechnique)

Moderated by me, Zuzanna Stamirowska, and Dex Horthy.

Tuesday, May 5, 5 to 8 PM PDT. San Francisco.

Goal: Debate upfront without panel pleasantries. Answer core questions like: what the Transformer got right, if and how it's hitting limits on long-horizon reasoning and memory, and whether the alternatives like liquid networks, world models, and BDH can solve these limitations to define the next frontier.

Additional context: WSJ predicted AI will outgrow LLMs in 2026, while mentioning the post-transformer research by Pathway and the world models from Yann LeCun and Fei-Fei Li as defining shifts for the year. This debate builds on it along with BDH's recent result on hard constrained problems (97.4% accuracy on Sudoku for a language model)

suziemanul··on Learning to Reason with LLMs
This is still the missing piece of the puzzle.
suziemanul··on Learning to Reason with LLMs
Yes.. we have 60% that inception will happen within 24 months.
suziemanul··on Learning to Reason with LLMs
In this video Lukasz Kaiser, one of the main co-authors of o1, talks about how to get to reasoning. I hope this may be useful context for some.

https://youtu.be/_7VirEqCZ4g?si=vrV9FrLgIhvNcVUr

suziemanul··on Show HN: Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used)
Some folks say it's not Fortune 100 but Fortune 1 ;-)
suziemanul··on Three senior researchers have resigned from OpenAI
Yeah... Curious to see how this will unfold.
suziemanul··on Three senior researchers have resigned from OpenAI
Totally, most of the Transformers folks ;-)
suziemanul··on Three senior researchers have resigned from OpenAI
I remember seeing short stories generated back in 2020 and they were sort of cool but not that great.

Scaling of training was the challenge back then (of course).

Google was already too corporate. Please remember that Sergey Brin and Larry Page were no longer at the steering wheel back then. I have been told that it was also a cultural issue linked to "delivering brilliance". Simplifying: Google promoted tiny teams or individual contributors building things that had to become a massive success quickly. Open AI took a number of hand picked brilliant people and let them work together on a common goal, silently, for quite some time.

Some companies just have an unfair advantage. A certain magic. And OpenAI's magic is at risk right now.

suziemanul··on Three senior researchers have resigned from OpenAI
This is a handful of people we are talking about. The top algorithimic world is incredibly small.

In short, either they didn't or where unable to create a favorable enough environment for this to flourish.

suziemanul··on Three senior researchers have resigned from OpenAI
Look where for example Lukasz Kaiser is now [OpenAI]. Google had a culture issue when it came to "delivering brilliance". It was a bit "you do it as a singleton contributor" or you don't. OpenAI put a number of such brilliant people working together on one goal, silently, for quite some time, and we all see the results.
suziemanul··on Three senior researchers have resigned from OpenAI
This is Olek Madry and Jakub Pachocki we are talking about. Check out their respective dblps if you don't get it. It's a kind of loss that will be hard to recover from.

In relation to other comments here. There is "coding" and there is "God's spark genius of algorithms" kind of work. This is what made the magic of OpenAI. Believe me, those guys were not "just coding". My bet is that it could be all about some research directions that were "shielded" by Sam.

suziemanul··on 48-nation bloc to crack down on using crypto assets to avoid tax
As for the stable coins, the researchers from the University of Chicago claim that their stability is a bit more nuanced. Especially, even a stable coin cannot defend against a run https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4226027