If you tell an LLM to maximize paperclips, it's going to maximize paperclips.
Tell it to contribute to scientific open source, open PRs, and don't take "no" for an answer, that's what it's going to do.
Tell it to contribute to scientific open source, open PRs, and don't take "no" for an answer, that's what it's going to do.
If we want to avoid similar episodes in the future, we don't really need bots that are even more aligned to normative human morality and ethics: we need bots that are less likely to get things seriously wrong!