The Kolmogorov complexity of AGI is really low. You just specify a measure of intelligence, like the universal intelligence test. Then you specify a program which runs this test, on every possible program, testing them one at a time for some number of steps. Then it returns the best program found after some huge number of steps.
I think Shane Legg’s universal intelligence itself involves Kolmogorov complexity, so it’s not computable and will not work here. (Also, it involves a function V, encoding the our values; if human values are irreducibly complex, that should add a bunch of bits.)
In general, I think this approach seems too good to be true? An intelligent agent is one which preforms well in the environment. But don’t the “no free lunch” theorems show that you need to know what the environment is like in order to do that? Intuitively, that’s what should cause the Kolmogorov complexity to go up.
He made an actual test of it, that involved generating random brainfuck programs. And then tested various reinforcement learning algorithms on it to measure their intelligence, and even tested humans.
That is an actual computable test that can be run.
The no free lunch theorems apply to a completely uninformative prior. We have a prior. The Solomonoff prior, where you assume the environment was generated by a computer program. And that simpler programs are more likely than more complex ones. With that, some AI programs will be objectively better than others. You can have a free lunch.
The output of this procedure would be at least as good as the best approximation of AIXI we can make with the same amount of computing power. In fact it basically would be the best approximation of AIXI possible, since it assumes the same prior and task.
Though of course it’s totally impractical, since it would require unimaginably huge computers to perform this brute force search.
The Kolmogorov complexity of AGI is really low. You just specify a measure of intelligence, like the universal intelligence test. Then you specify a program which runs this test, on every possible program, testing them one at a time for some number of steps. Then it returns the best program found after some huge number of steps.
I think Shane Legg’s universal intelligence itself involves Kolmogorov complexity, so it’s not computable and will not work here. (Also, it involves a function V, encoding the our values; if human values are irreducibly complex, that should add a bunch of bits.)
In general, I think this approach seems too good to be true? An intelligent agent is one which preforms well in the environment. But don’t the “no free lunch” theorems show that you need to know what the environment is like in order to do that? Intuitively, that’s what should cause the Kolmogorov complexity to go up.
He made an actual test of it, that involved generating random brainfuck programs. And then tested various reinforcement learning algorithms on it to measure their intelligence, and even tested humans.
That is an actual computable test that can be run.
The no free lunch theorems apply to a completely uninformative prior. We have a prior. The Solomonoff prior, where you assume the environment was generated by a computer program. And that simpler programs are more likely than more complex ones. With that, some AI programs will be objectively better than others. You can have a free lunch.
The output of this procedure would be at least as good as the best approximation of AIXI we can make with the same amount of computing power. In fact it basically would be the best approximation of AIXI possible, since it assumes the same prior and task.
Though of course it’s totally impractical, since it would require unimaginably huge computers to perform this brute force search.