Hacker News Reader

Comment by yeck | Hacker News Reader

If you are going in this direction, please consider working on AI safety and alignment problems. Things like interpretability and corrigibility are keys to creating systems we can understand and "steer". Both are necessary if we hope to be able to use AI successfully. Success means safely and effectively (actually does what you expect/want).

Some resources:

- https://80000hours.org/podcast/episodes/chris-olah-interpret...

- https://www.fast.ai/ (haven't used it, but heard good things)

- https://www.agisafetyfundamentals.com/ai-alignment-curriculu...

View on HN

I'm curious if the downvoters can elaborate on why they consider this a bad direction. From my perspective, it seems like AI safety may be an under-utilized field that is more ripe for a beginner than the more mature aspects. E.g., DARPA has committed large sums of money ML assurance. IMO AI, particularly in safety-critical domains, will need such measures to build a sufficient level of trust before wide-spread adoption.

I didn't feel like the parent comment was worth downvoting, and I share most of your perspectives. I agree that AI safety is ripe for contributions, but I'd personally argue AI safety is a very wide and encompassing topic that requires a good foundational understanding of AI. In the context of ML, AI safety is applicable to the entire process; from dataset creation, training, utilization, etc.

In regards to your comment on AI safety being "underutilized", my thoughts are that it's just simply difficult to do. Let's put aside all the difficulties of training, verification, etc and just look at the data problem.

If you wish to make certain that your system meets some given AI safety standard, then you must somehow prove two things: the data the model ingests when deployed will always return the correct response and that the dataset composes/generalizes the data the model will ingest when deployed. For simple problems, this may be doable. For complex, multidimensional problems wherein the dataset must only hope to generalize the complex input it will encounter during deployment, this may be next to infeasible.

I'm definitely getting off topic here, but bias of all kinds exists even human operated systems eg car. I can't say I've ever seen a firetruck stopped on the highway before, but perhaps I'd know what it is and how to avoid it. If a dataset does not contain that event, how can we be certain an AI system would understand? I'm not sure if it's possible to create a dataset that will be without bias in the case of complex problems, but I'm certain we can create one that's performant at driving than I. So the questions of "how safe is enough?", then proving/demonstrating that safety, and more are particularly open topics. I enjoy making the point that there is the lack of rigorous standards for humans, as we hold computers to far higher standards, but ML models probabilistically navigate decisions similar to us.

I'm sure this reply could extend further, but this and more are my defense of why I believe AI safety is a wide topic. None of the above should dissuade beginners from exploring the subtopic, but it's certainly not something you'd be able to learn first without strong, foundational context.

I think we probably come to some of the same conclusions, but I'm approaching this problem slightly differently.

To use your car example: say I'm driving in front of a park where there are lots of parked cars lining the street. Then a ball rolls out into the street from between two parked cars. I may have never personally seen a child run into the street from between two parked cars before, but I can infer (i.e., imagine) that from the context of the scenario. So I slow waaaay down in case that event happens. I don't need to see all edge case to still cover an awful lot of them.

I'm not sure AI is to that point (yet). There are some arguments for approaches like reinforcement learning that say they perform quite well on unseen edge cases from past learning. But when the stakes are high, I'm not sure that is good enough.

(And regarding the 'it only has to be slightly better than the average human' counterpoint): I disagree. I think one of the reasons that we are comfortable with sharing the road with other ape-driven vehicles is that we have a theory of mind and can intuit what someone else is thinking and are able to 'imagine' their course of action. We've evolved to have this sense. We do not, however, have the ability to intuit what a computer will do because it 'evolved' under very different circumstances. So our intuitions about whether or not to trust it may be out of whack with whether or not it performs better. And, like it or not, the policy that governs if AI-controlled cars is legal will be highly dependent on public trust.

Yeah, disappointing to see this downvoted, and without even any down voters attempting to make a case for why this isn't a "good" answer.

I think that this part of the field is going to see a lot of relative growth as the broader public and governments become increasingly concerned. And for those who are not concerned with safety, it is not unreasonable to expect that capability progress could become bottlenecked by lack of model interpretability. Speculative, sure, but if we find that more data and transforms hits diminishing returns, it could prove significant to know what is going on inside the models. Maybe extracting and composing circuits is essential for AGI? Who knows?

I like this very much, i am hearing the word responsible AI alot these days and there will be lot of need for some one to understand the AI alignment problem and help companies achieve it.

How did cats and dogs solve the human alignment problem?

Reply on news.ycombinator.com