For the first, the succinct argument for correctness is to consider the details of key barriers.
Imagine the machine is trying to convince a human it doesn’t know to do something in favor of the . machine. More and more intelligence you can model as allowing the machine to consider an ever wider search space of possible hidden state for the human or messages it can emit.
But none of this does more than marginally improve the pSuccess. For this task I will claim the odds of success with human intelligence are 10 percent, and with infinite intelligence, 20 percent. It takes logarithmically more compute to approach 20 percent.
Either way the machine is probably going to fail. I am claiming there are thousands of real world tasks on the way to conquering the planet with such high pFail.
The way the machine wins is to have overwhelming force, same way you win any war. And that real world force has a bunch of barriers to obtaining.
For the second, again, debates are one thing. Taking costly action (delays, nuclear war) is another. I am saying it is irrational to take costly actions without direct evidence.
For the first, the succinct argument for correctness is to consider the details of key barriers.
Imagine the machine is trying to convince a human it doesn’t know to do something in favor of the . machine. More and more intelligence you can model as allowing the machine to consider an ever wider search space of possible hidden state for the human or messages it can emit.
But none of this does more than marginally improve the pSuccess. For this task I will claim the odds of success with human intelligence are 10 percent, and with infinite intelligence, 20 percent. It takes logarithmically more compute to approach 20 percent.
Either way the machine is probably going to fail. I am claiming there are thousands of real world tasks on the way to conquering the planet with such high pFail.
The way the machine wins is to have overwhelming force, same way you win any war. And that real world force has a bunch of barriers to obtaining.
For the second, again, debates are one thing. Taking costly action (delays, nuclear war) is another. I am saying it is irrational to take costly actions without direct evidence.