Also worth mentioning this concept “Value learning” is called out specifically in Nick Bostrom’s book, Superintelligence, with the use of the envelope puzzle which goes a little something like this; “Suppose we write down a description of a set of values on a piece of paper. We fold that paper and put it in a sealed envelope. We then create an agent with human-level general Intelligence and give it the following final goal; Maximize the realisation of the values described in the envelope.”
Also worth mentioning this concept “Value learning” is called out specifically in Nick Bostrom’s book, Superintelligence, with the use of the envelope puzzle which goes a little something like this; “Suppose we write down a description of a set of values on a piece of paper. We fold that paper and put it in a sealed envelope. We then create an agent with human-level general Intelligence and give it the following final goal; Maximize the realisation of the values described in the envelope.”