Consider marginal utility. Many people are working on AI, machine learning, computational psychology, and related fields. Nobody is working on preference theory, formal understanding of our goals under reflection. If you want to do interesting research and if you have the background to advance either of those fields, do you think the world will be better off with you on the one side or on the other?
Maybe that’s true, but that’s a separate point. “Let’s work on preference theory so that it’ll be ready when the AI catches up” is one thing—tentatively, I’d say it’s a good idea. “Let’s campaign against anybody doing AI research” seems less useful (and less likely to be effective.)
Consider marginal utility. Many people are working on AI, machine learning, computational psychology, and related fields. Nobody is working on preference theory, formal understanding of our goals under reflection. If you want to do interesting research and if you have the background to advance either of those fields, do you think the world will be better off with you on the one side or on the other?
Maybe that’s true, but that’s a separate point. “Let’s work on preference theory so that it’ll be ready when the AI catches up” is one thing—tentatively, I’d say it’s a good idea. “Let’s campaign against anybody doing AI research” seems less useful (and less likely to be effective.)