Hacker News Reader

saladtoeson Claude 4 System Card

Agreed on CaMeL as a promising direction forward. Guardrails may not get 100% of the way but are key for defense in depth, even approached like CaMeL currently fall short for text to text attacks, or more e2e agentic systems.

saladtoeson Claude 4 System Card

https://www.lakera.ai/blog/claude-4-sonnet-a-new-standard-fo...

These LLMs still fall short on a bunch of pretty simple tasks. Attackers can get Claude 4 to deny legitimate requests easily by manipulating third party data sources for example.

saladtoeson Brex’s Prompt Engineering Guide

I've been playing Gandalf in the last few days, it does a great job at giving an intuition for some of the subtleties of prompt engineering: https://gandalf.lakera.ai

Thanks for putting this together!

saladtoeson Scientists Increasingly Can’t Explain How AI Works

Is that really a universal fact? In any case, my statement goes in the opposite direction: is a reliable system necessarily "well understood", in the sense that it can explain its decisions? Most complex systems powering our lives cannot tell us anything about how they made those decisions.

AI systems add a layer of complexity. Even if you can explain a decision well on your training data, I seriously doubt you will be able to still provide reasonable explanations in completely out of distribution data.

saladtoeson Scientists Increasingly Can’t Explain How AI Works

Explainability is not a given in many more traditional complex systems. Decisions are often an aggregation of a large number of signals, and one can often not conceive of a single intuitive explanation for the system's decisions.

A lot is expected of AI systems today, from fairness (how do we even define that?) to universality. In my view we need to develop a practical understanding of what it means to build the system we have in mind: do I understand where I want my system to perform, and do I have the tools to assess whether I am getting there? Interpretability is orthogonal to all of this.

I would much rather have a well tested system, accompanied by online monitoring to detect unusual inputs in an ever-changing data distribution and notify when updates are needed or a human needs to take control, than an unreliable system that is great at providing explanations.

saladtoeson [dead]

Interesting, thanks for sharing! Somehow I'm not surprised. My experience building systems for real world applications is that choosing a simple CNN is usually the way to go, and it's all in the data. Choosing more complex model classes also comes at a cost, higher variance and more likelihood of issues related to robustness and generalization.

saladtoeson [dead]

“The latter being when the training or test data follows a different distribution to the in-operation data”

This form of ML bug is the most challenging to catch. The true in-operation distribution is often unknown which makes testing for such bugs a very challenging problem. Any thoughts on this?

saladtoeson Seeing the forest for the trees: A more disciplined approach for AI (2021)

Spot on, the EU seems to agree: https://techcrunch-com.cdn.ampproject.org/c/s/techcrunch.com...

saladtoeson They Write the Right Stuff (2021)

Oh no!! I really wanted to know how they develop their AI so reliably :). This is a massive issue today. Hope we develop tools and processes to get us there soon.”