We would want the AI we build to ultimately share our values, so that it can work as an extension of our will. It does not look promising to write down a long list of everything we care about. It looks more promising to leverage the AI’s own intelligence to learn about our values and what our preferences are.
This looks like a nice intro to a dystopian sci-fi movie.