Hacker News Reader

How can you control the creation of code?

This news sounds like "donating to keep software bug-free". If someone wants or even accidentally, they can create buggy code or "evil" AI.

View on HN

You can donate to keep software bug-free by funding research into safer programming languages (e.g., Rust), tools for formal verification and static analysis, testing frameworks, software engineering methodology, etc. Sure, someone could always ignore all the research advances and just write unsafe, untested C code. But for the most part, people want to eliminate bugs, so as the techniques for doing so mature, they will tend to become adopted in practice and certain types of bugs will become much less frequent.

Similarly, there are technical research questions related to how to design an intelligent agent that is "controllable", in that it acts to achieve its given goals but not to such an extent that it would resist being switched off or reprogrammed with new goals. Making progress on answering these questions doesn't guarantee that any given developer will use whatever techniques are discovered. But insofar as pretty much everyone, even evil masterminds, wants to maintain control over their AIs, the availability of techniques for 'safe' AI will at least decrease the likelihood that a powerful AI is built without any safeguards.

Perhaps the incentive here is to educate coders as to what patterns to avoid which will have an inclination toward "evil" AI. This won't stop others for whom this is the goal.

Reply on news.ycombinator.com