A rigorous theory of valence wouldn’t involve cultural context, much as a rigorous theory of electromagnetism doesn’t involve cultural context.
Cultural context may matter a great deal in terms of how to build a friendly AGI that preserves what’s valuable about human civilization—or this may mostly boil down to the axioms that ‘pleasure is good’ and ‘suffering is bad’. I’m officially agnostic on whether value is simple or complex in this way.
One framework for dealing with the stuff you mention is Coherent Extrapolated Volition (CEV)- it’s not the last word on anything but it seems like a good intuition pump.
And I guess I’m saying that the sooner we think about these sorts of things the better off we’ll be. Going for pleasure good/suffering bad reduced the mindset of AI to about 2 years old. Cultural context gives us a sense of maturity Valence or no.
A rigorous theory of valence wouldn’t involve cultural context, much as a rigorous theory of electromagnetism doesn’t involve cultural context.
Cultural context may matter a great deal in terms of how to build a friendly AGI that preserves what’s valuable about human civilization—or this may mostly boil down to the axioms that ‘pleasure is good’ and ‘suffering is bad’. I’m officially agnostic on whether value is simple or complex in this way.
One framework for dealing with the stuff you mention is Coherent Extrapolated Volition (CEV)- it’s not the last word on anything but it seems like a good intuition pump.
And I guess I’m saying that the sooner we think about these sorts of things the better off we’ll be. Going for pleasure good/suffering bad reduced the mindset of AI to about 2 years old. Cultural context gives us a sense of maturity Valence or no.