It seems that consciousness is one of the essential things necessary to characterize the moral status of an agent.
It seems that we have very little chance to solve the AI safety problem.
We don’t know if the first AGI will be conscious.
Since we are in a certain sense doomed, the future consists of the following 2 cases:
Either the first AGI is not sentient and never becomes sentient.
Or the first AGI acquires sentience at a given moment.
Either the valence of this AGI is positive
Or the valence of this AGI is negative
Either the valence is mixed
Or the AGI reprograms itself to experience a positive valence (instrumental convergence if its well-being is included in the utility function?)
I have no idea which scenario is the most desirable. The uncertainty is multiplied by considering the AGI’s superior capabilities, and its ability to multiply. So perhaps we could see it as a utility monster. Therefore, the following questions seem very important:
(1) What do you think is the most likely scenario and (2) the most desirable scenario?
(3|2) Conditional on the scenario that seems most desirable to you, is there a way to steer the future in that direction?
It’s becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman’s Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first. The thing I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990′s and 2000′s. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I’ve encountered is anywhere near as convincing. I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there’s lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order. My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar’s lab at UC Irvine, possibly. Dr. Edelman’s roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
“and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first”
The language models like GPT3 appear to be unstoppable, and are much more versatile than the models of other modalities.
So maybe a machine with higher order consciousness will come first?
The difference between primary consciousness and higher order consciousness is not clear to me. Is it the same thing as access consciousness and meta cognition?
I just discovered this debate thanks to a YouTube recommendation from the Institute for the Future of Life. I find the formulation of the question by Anderw Serazin very well put.
The Yes Team: We would identify with it better, and it would help with creativity. You would want your digital assistant to be able to help you by modeling you, it seems necessary to model the human.
No team: It’s easier to deal with IA safety without it, the possibility to create infinite suffering is bad, unpleasant task in the world are better done by non-conscious agent.
I think the no team is right.
Interesting: during the questions, Yoshua Bengio advocates decoupling moral status from subjective experience (Moral status: would be the role in society of the entity !?). And then he proposes the following taxonomy of the concept of consciousness:
Subjective experience: trivially already incorporated in modern deep learning because each neural network has learned its own representation of the world.
self-awareness: useful notion for the agent who moves in the world
emotion: already present in reinforcement learning in a primitive way.