I like your comment, but I do want to comment this:
Arguments that “we’re training it on human data so it will care about our values above zero” are extremely speculative. They could be true, but betting the future of humanity on it without thinking it through seems very, very foolish.
has evidence against it, fortunately for us.
I summarize the evidence for the pretty large similarities between the human brain and current DL systems, and this allows us to transport insights from AI into neuroscience, and vice versa here:
But the point here is that one of the lessons from AI that is likely to transfer over to human values is that the data matters way more than the algorithm, optimizer, architecture, or hyperparameter choices.
I don’t go as far as this link does in claiming that the it in AI models is the data set, but I think a weaker version of this is basically right, and thus the bitter lesson holds for human values too:
I like your comment, but I do want to comment this:
has evidence against it, fortunately for us.
I summarize the evidence for the pretty large similarities between the human brain and current DL systems, and this allows us to transport insights from AI into neuroscience, and vice versa here:
https://x.com/SharmakeFarah14/status/1837528997556568523
But the point here is that one of the lessons from AI that is likely to transfer over to human values is that the data matters way more than the algorithm, optimizer, architecture, or hyperparameter choices.
I don’t go as far as this link does in claiming that the it in AI models is the data set, but I think a weaker version of this is basically right, and thus the bitter lesson holds for human values too:
https://nonint.com/2023/06/10/the-it-in-ai-models-is-the-dataset/
Other than that quote, I basically agree with the rest of your helpful comment here.