I tend to agree with most of what has been said. The only difference is that some workflows need to be deterministic otherwise the penalty for failure is high. In that case, AI is helpful in searching through a space, but something orthogonal needs to verify its output.
LinkedIn is a long-form billboard not a blogging site. It’s meant for getting attention, not for thought provoking discussion. Or at least, I’ve not really seen honest discourse on any recent social platform. There are plenty of reasons to be on a billboard, but most billboards are boring.
This article is really two. One that describes RL. The other is how they applied it. The former was quite helpful because it demystified much of the jargon that I find in AI. All branches of science have jargon. I find the AI ones especially impenetrable.
I learned recursion in Pascal and had a ball implementing innumerable algorithms purely in recursive structures. The code was often more compact and simpler to explain — once you understood induction. I agree that combinators are helpful, but they’re not universal. Anyway, we’re probably violently agreeing.
Tail recursion is beautifully deep and simple. It (and as a corollary CPS) makes clear what state matters to your loop (and function) and avoids extraneous variables in loops as well as implicit unneeded state in functions. It also makes it easier to show the correctness of your loops. Sure, there are other functional language constructs like fold and map, if your problem is amenable to them. Tail recursion is more general and simpler.
This is a tragedy. Our pre-eminence as a scientific and industrial powerhouse that really began post WWII is now disintegrating because of the actions of a few. The funding being pulled from Terence Tao and his institute without due process is not the start, it's merely one casualty among many that began at the start of this administration. This is like cutting one's nose to spite one's face.
Fantastic post. I became a manager in the style of the Steve Jobs philosophy— take one of your best individual contributors and make them oversee the work of others. Professional managers didn’t work out well for Apple, but the approach of letting growing leaders lead did. This post misses out on the important point to rely on your instincts because that’s why you’ve been put in that position.
A beautiful and simple explanation of formal specs. I especially like the remark that a spec can be incorrect. The art of building correct systems is just in showing the implementation is correct but also choosing a spec that matches our intuitive understanding of the system.
The AI scaling that went on for the last five years is going to be very different from the scaling that will happen in the next ten years. These models have latent capabilities that we are racing to unearth. IMO is but one example.
There’s so much to do at inference time. This result could not have been achieved without the substrate of general models. Its not like Go or protein folding. You need the collective public global knowledge of society to build on. And yes, there’s enough left for ten years of exploration.
More importantly, the stakes are high. There may be zero day attacks, biological weapons, and more that could be discovered. The race is on.
While their growth is faster and technology different, the atmosphere feels very much like AWS back in 2014. I stayed for 8 years because I enjoyed it so much.
This talk is deceptively simple. The most sage advice that founders routinely forget is what concrete idea are you going to implement and why do you think it will work? There has be a way to invalidate your idea and as a corollary you must have the focus to collect the data and properly invalidate it.
Cool post. We did a similar evaluation for document segmentation using the DocLayNet benchmark from IBM: https://ds4sd.github.io/icdar23-doclaynet/task/ but on modern document OCR models like Mistral, OpenAI, and Gemini. And what do you know, we found similar performance -- DETR-based segmentation models are about 2x better.
Wow! What if we all did this? What is the closure of the feature set that ChatGPT can imagine for your product. Is it one that is easy for ChatGPT to use? Is it one that is sound and complete for your use cases? Is it the best that you can build had you had clear requirements upfront?
Its funny. Generally, people in the startup world frowned on this bill because of the cuts to essential services. Nonetheless, we’re thrilled about the expansion of QSBS. Perhaps there’s always a silver lining.
Fine tuning isn’t for everything but certainly makes it easy to build models for special purposes, eg metadata extraction. Happy to lose some capability in another domain for that, eg Pokémon. The headline is a bit too general.
Legend. I still remember first putting my hands on a Mac, and the joy of computing that ensued in high school. I could get lost in the computer for days. Thank you, Bill.
Verification is going to be an increasing problem with AI. Most of the work will be in verifying the incredible guesses that AI make. In some cases, it'll be important to easily ferret out the false positive, and in others, it'll be critical to ensure there are no false negatives. In science especially, our focus and reward structure will need to be on proper and sound verification.
I would argue that this is already true in roles where one supervises the work of another person with skill. Great leaders, for example, were once practitioners. Over time their skills may fade, but their judgment makes them effective and able to the scale their impact.
The scary part of this is that the bad guys are doing the same thing. They’re looking for zero day exploits, and their ability to find them just got better. More importantly, it’s now almost automated. While the arms race will always continue, I wonder if this change of speed hurts the good guys more than the bad guys. There are many of these, and they take time to fix.
AWS also had customers that had petabytes of data in Redshift for analysis. The conversation is missing a key point: DuckDB is optimizing for a different class of use cases. They’re optimizing for data science and not traditional data warehousing use cases. It’s masquerading as size. Even for small sizes, there are other considerations: access control, concurrency control, reliability, availability, and so on. The requirements are different for those different use cases. Data science tends to be single user, local, and lower availability requirements than warehouses that serve production pipelines, data sharing, and so on. I also think that DuckDB can be used for those, but not optimized for those.