Little thought experiment with flavors of Newcomb and Berry’s Paradox:
I have the code of an ASI in front of me, translated into C along with an oracle to a high-quality RNG. This code is N characters. I want to compete with this ASI at the task of writing a 2N-character C code that halts and prints a very large integer. Will I always win?
Sketch of why: I can write my C code to simulate the action of the ASI on a prompt like “write a 2N-character C code that halts and prints the largest integer” using every combination of possible RNG calls and print the max + 1 or something.
Sketch of why not: The ASI can make us both lose by “intending” to print a non-halting program if it is asked to. There might be probabilistic approaches for the ASI as well, where it produces a non-halting program with some chance. If I can detect this in the simulations, I might be able to work around this and still beat the ASI.
My guess is “no” because both of you would die first. In the context of “largest numbers” 10^10^100 is baby’s first step, but is still a number with more digits than you will ever succeed in printing.
In principle the “you” in this scenario could be immortal with unbounded resources and perfect reliability, but then we may as well just suppose you are a superintelligence smarter than the AI in the problem (which isn’t looking so ‘S’ anymore).
To clarify: we are not running any programs, just providing code. In a sense, we are competing at the task of providing descriptions for very large numbers with an upper bound on the size of the description (and the requirement that the description is computable).
One problem is that “every possible RNG call” may be an infinite set. For a really simple example, a binary {0,1} RNG with program “add 1 to your count if you roll 1 and repeat until you roll 0″ has infinitely many possible rolls and no maximum output. It halts with probability 1, though.
If you allow the RNG to be configured for arbitrary distributions then you can have it always return a number from such a distribution in a single call, still with no maximum.
Oops, yeah the written programs are supposed to be deterministic. The point of mentioning the RNG was to handle the fact that an AI might derive its performance from a strong random number generator, which a C code can’t emulate.
Little thought experiment with flavors of Newcomb and Berry’s Paradox:
I have the code of an ASI in front of me, translated into C along with an oracle to a high-quality RNG. This code is N characters. I want to compete with this ASI at the task of writing a 2N-character C code that halts and prints a very large integer. Will I always win?
Sketch of why: I can write my C code to simulate the action of the ASI on a prompt like “write a 2N-character C code that halts and prints the largest integer” using every combination of possible RNG calls and print the max + 1 or something.
Sketch of why not: The ASI can make us both lose by “intending” to print a non-halting program if it is asked to. There might be probabilistic approaches for the ASI as well, where it produces a non-halting program with some chance. If I can detect this in the simulations, I might be able to work around this and still beat the ASI.
My guess is “no” because both of you would die first. In the context of “largest numbers” 10^10^100 is baby’s first step, but is still a number with more digits than you will ever succeed in printing.
In principle the “you” in this scenario could be immortal with unbounded resources and perfect reliability, but then we may as well just suppose you are a superintelligence smarter than the AI in the problem (which isn’t looking so ‘S’ anymore).
To clarify: we are not running any programs, just providing code. In a sense, we are competing at the task of providing descriptions for very large numbers with an upper bound on the size of the description (and the requirement that the description is computable).
Oh, I see that I misread.
One problem is that “every possible RNG call” may be an infinite set. For a really simple example, a binary {0,1} RNG with program “add 1 to your count if you roll 1 and repeat until you roll 0″ has infinitely many possible rolls and no maximum output. It halts with probability 1, though.
If you allow the RNG to be configured for arbitrary distributions then you can have it always return a number from such a distribution in a single call, still with no maximum.
Oops, yeah the written programs are supposed to be deterministic. The point of mentioning the RNG was to handle the fact that an AI might derive its performance from a strong random number generator, which a C code can’t emulate.