Imagine the best possibility (for humans) consistent with today’s physics. Imagine the best (for humans) mathematical facts.
No you don’t. Penroses theory is totally abstract computability theory. If it were true, then so what? The best for humans facts are something like “alignment is easy, FAI built next week”. This only works if penrose somehow got a total bee in his bonnet about uncomputability, it greatly offended his sensibilities that humans couldn’t know everything. Even though we empirically don’t. Even though pragmatic psycological bounds are a much tighter constraint than computability. In short, your theory of “motivated cognition” doesn’t help predict much. Because you need to assume penroses motivations are just as wacky.
Also, you seem to have slid from “motivated cognition works to produce true beliefs/optimize the world” to the much weaker claim of “some people use motivated cognition, you need to understand it to predict there behavior”. This is a big jump, and feels mote and bailey.
No you don’t. Penroses theory is totally abstract computability theory. If it were true, then so what?
I have to say, I can definitely see it, and I even think it’s obvious why. Penrose’s theory assures that consciousness is the special phenomenon we feel it is, it assures the unity of the mind through a special unity of consciousness, it assures human exceptionality over any automata (we would be, in a relevant sense, NOT automata), and it rescues the idea of (libertarian) ‘free will’ in the most satisfactory way possible given the basic commitments coming from physics (and this distinguishes us from mere programmed automata).
I always thought that I would prefer his theory to be true, I dont feel bad about it but it’s true.
It is pretty obvious that Penrose has pre-commitments to human exceptionality and the specialness of consciousness (and he has shown them pretty explicitly, by appealing to incompleteness theorems against AI and saying that “it can’t just be natural selection”), and that’s what motivates the theory. It’s not a coincidence.
Suppose human brains turned out to be hypercomputers. (They really really aren’t) But imagine the world where they were.
We describe what is going on with equations. It’s some complicated thing involving counterpolarized pseudoquarks or something. We can recreate this phenomena in the lab. Recent experiments with sheets of graphine in liquid xenon produce better hypercomputers.
A halting oracle doesn’t seem to contain magic “free will” stuff. It’s behaviour is formally specified and mathematically determined. A quantum coin toss can already make random data, something no turing machine can do. (Yet the universe is computable, because a reality goes both ways) Is a quantum coin conscious? No, it’s just a source of randomness.
As soon as the magic mysterious essence of consciousness that Penrose hopes for is actually discovered, it will stop being magical and mysterious, and will be demoted to the dull catalogue of common things.
A halting oracle doesn’t seem to contain magic “free will” stuff
I know, although that’s really what libertarian FW entails in physicalist terms. In fact, funnily enough, I read yesterday Turing’s paper Computer Machinery and Intelligence (mostly just to be amused at his theological and ESP counter-arguments) and he literally said that some people describe machines with randomness oracles as having ‘free will’.
Regardless, a major point of Penrose’s theory is that the mind works as a cohesive whole thanks to a quantum superposition of the brain, it’s a much more remarkable feature than just having any form of LFW. I don’t even get the appealing of LFW anymore.
Well, there’s one thing that I miss of the idea of having LFW (being an indeterministic system). Whenever I think about alternate history scenarios, no matter how plausible they look, the actual probabilities of having happened are 0, and that’s just boring.
Also, you seem to have slid from “motivated cognition works to produce true beliefs/optimize the world” to the much weaker claim of “some people use motivated cognition, you need to understand it to predict there behavior”. This is a big jump, and feels mote and bailey.
Most parts of the post are explicitly described as “this is how motivated cognition helps us, even if it’s wrong”. Stronger claims return later. And your weaker claim (about predicting people) is still strong and interesting enough.
No you don’t. Penroses theory is totally abstract computability theory. If it were true, then so what? The best for humans facts are something like “alignment is easy, FAI built next week”. This only works if penrose somehow got a total bee in his bonnet about uncomputability, it greatly offended his sensibilities that humans couldn’t know everything. Even though we empirically don’t. Even though pragmatic psycological bounds are a much tighter constraint than computability. In short, your theory of “motivated cognition” doesn’t help predict much. Because you need to assume penroses motivations are just as wacky.
There I talk about the most interesting possibility in the context of physics and math, not Alingment. And I don’t fully endorse Penrose’s “motivation”, even without Alingment his theory is not the most interesting/important thing to me. I treat Penrose’s theory as a local maximum of optimism, not the global maximum. You’re right. But this still helps to remember/highlight his opinions.
I’m not sure FAI is the global maximum of optimism too:
There may be things that are metaphysically more important. (Something about human intelligence and personality.)
We have to take facts into account too. And facts tell that MC doesn’t help to avoid death and suffering by default. Maybe it could help if it were more widespread.
Those two factors make me think FAI wouldn’t be guaranteed if we suddenly learned that “motivated cognition works (for the most part)”.
No you don’t. Penroses theory is totally abstract computability theory. If it were true, then so what? The best for humans facts are something like “alignment is easy, FAI built next week”. This only works if penrose somehow got a total bee in his bonnet about uncomputability, it greatly offended his sensibilities that humans couldn’t know everything. Even though we empirically don’t. Even though pragmatic psycological bounds are a much tighter constraint than computability. In short, your theory of “motivated cognition” doesn’t help predict much. Because you need to assume penroses motivations are just as wacky.
Also, you seem to have slid from “motivated cognition works to produce true beliefs/optimize the world” to the much weaker claim of “some people use motivated cognition, you need to understand it to predict there behavior”. This is a big jump, and feels mote and bailey.
Late reply, sorry if it’s a bother.
I have to say, I can definitely see it, and I even think it’s obvious why. Penrose’s theory assures that consciousness is the special phenomenon we feel it is, it assures the unity of the mind through a special unity of consciousness, it assures human exceptionality over any automata (we would be, in a relevant sense, NOT automata), and it rescues the idea of (libertarian) ‘free will’ in the most satisfactory way possible given the basic commitments coming from physics (and this distinguishes us from mere programmed automata).
I always thought that I would prefer his theory to be true, I dont feel bad about it but it’s true.
It is pretty obvious that Penrose has pre-commitments to human exceptionality and the specialness of consciousness (and he has shown them pretty explicitly, by appealing to incompleteness theorems against AI and saying that “it can’t just be natural selection”), and that’s what motivates the theory. It’s not a coincidence.
Suppose human brains turned out to be hypercomputers. (They really really aren’t) But imagine the world where they were.
We describe what is going on with equations. It’s some complicated thing involving counterpolarized pseudoquarks or something. We can recreate this phenomena in the lab. Recent experiments with sheets of graphine in liquid xenon produce better hypercomputers.
A halting oracle doesn’t seem to contain magic “free will” stuff. It’s behaviour is formally specified and mathematically determined. A quantum coin toss can already make random data, something no turing machine can do. (Yet the universe is computable, because a reality goes both ways) Is a quantum coin conscious? No, it’s just a source of randomness.
As soon as the magic mysterious essence of consciousness that Penrose hopes for is actually discovered, it will stop being magical and mysterious, and will be demoted to the dull catalogue of common things.
I know all of that, believe me.
I know, although that’s really what libertarian FW entails in physicalist terms. In fact, funnily enough, I read yesterday Turing’s paper Computer Machinery and Intelligence (mostly just to be amused at his theological and ESP counter-arguments) and he literally said that some people describe machines with randomness oracles as having ‘free will’.
Regardless, a major point of Penrose’s theory is that the mind works as a cohesive whole thanks to a quantum superposition of the brain, it’s a much more remarkable feature than just having any form of LFW. I don’t even get the appealing of LFW anymore.
Well, there’s one thing that I miss of the idea of having LFW (being an indeterministic system). Whenever I think about alternate history scenarios, no matter how plausible they look, the actual probabilities of having happened are 0, and that’s just boring.
Most parts of the post are explicitly described as “this is how motivated cognition helps us, even if it’s wrong”. Stronger claims return later. And your weaker claim (about predicting people) is still strong and interesting enough.
There I talk about the most interesting possibility in the context of physics and math, not Alingment. And I don’t fully endorse Penrose’s “motivation”, even without Alingment his theory is not the most interesting/important thing to me. I treat Penrose’s theory as a local maximum of optimism, not the global maximum. You’re right. But this still helps to remember/highlight his opinions.
I’m not sure FAI is the global maximum of optimism too:
There may be things that are metaphysically more important. (Something about human intelligence and personality.)
We have to take facts into account too. And facts tell that MC doesn’t help to avoid death and suffering by default. Maybe it could help if it were more widespread.
Those two factors make me think FAI wouldn’t be guaranteed if we suddenly learned that “motivated cognition works (for the most part)”.