One operationalization of “know” in this case is being able to accurately predict every move of the Go AI. This is a useful framing, because instead of a hard pass/fail criterion, we can focus on improving our calibration.
Now the success criterion might be:
You have to be able to attain a Brier score of 0 in predicting the moves of the best go bot that you have access to.
What’s missing are some necessary constraints.
Most likely, you want to prohibit the following strategies:
Running a second instance of the Go AI on the same position, and using as your prediction the move that instance #2 makes.
Manually tracing through the source code to determine what the output would be if it was run.
Memorizing the source code and tracing through it in your head.
Constraining the input moves to ones where every Go program would make the same move, then using the output of one of a different Go program as your input.
Corollary: you can’t use any automation whatsoever to determine what move to make. Any automated system that can allow you to make accurate predictions is effectively a Go program.
Overall, then you might just want to prohibit the use of Turing machines. However, my understanding is that this results in a ban on algorithms. I don’t have enough CS to say what’s left to us if we’re denied algorithms.
Here’s a second operationalization of “know.” You’re allowed to train up using all the computerized help you want. But then, to prove your ability, you have to perfectly predict the output of the Go program on a set of randomly generated board positions, using only the power of your own brain. A softer criterion is to organize a competition, where participants are ranked by Brier score on this challenge.
However, this version of the success criterion is just a harder version of being an inhumanly good Go player. Not only do you have to play as well as the best Go program, you have to match its play. It’s the difference between being a basketball player with stats as good as Michael Jordan’s, and literally being able to copy his every move in novel situations indefinitely.
Neither of these versions of the success criterion operationalization seems particularly interesting. Both are too restrictive to be relevant to AI safety.
Did you have a different operationalization in mind?
Here’s a second operationalization of “know.” You’re allowed to train up using all the computerized help you want. But then, to prove your ability, you have to perfectly predict the output of the Go program on a set of randomly generated board positions, using only the power of your own brain.
I was thinking more of propositional knowledge (well, actually belief, but it doesn’t seem like this was a sticking point with anybody). A corollary of this is that you would be able to do this second operationalization, but possibly with the aid of a computer program that you wrote yourself that wasn’t just a copy of the original program. This constraint is slightly ambiguous but I think it shouldn’t be too problematic in practice.
Did you have a different operationalization in mind?
The actual thing I had in mind was “come up with a satisfactory operationalization”.
A corollary of this is that you would be able to do this second operationalization, but possibly with the aid of a computer program that you wrote yourself that wasn’t just a copy of the original program. This constraint is slightly ambiguous but I think it shouldn’t be too problematic in practice.
I’m going to assume it’s impossible for me, personally, to outplay the best Go AI I have access to. Given that, the requirement is for me to write a better Go AI than the one I currently have access to.
Of course, that would mean that my new self-written program is now the best Go AI. So then I would be back to square one.
There are weaker computational machines than Turing machines, like regexes. But you don really care about that, you just want to ban automatic reasoning. I think it’s impossible to succeed with that constrain; Playing Go is hard, people can’t just read code that plays Go well and “learn from it.”
One operationalization of “know” in this case is being able to accurately predict every move of the Go AI. This is a useful framing, because instead of a hard pass/fail criterion, we can focus on improving our calibration.
Now the success criterion might be:
You have to be able to attain a Brier score of 0 in predicting the moves of the best go bot that you have access to.
What’s missing are some necessary constraints.
Most likely, you want to prohibit the following strategies:
Running a second instance of the Go AI on the same position, and using as your prediction the move that instance #2 makes.
Manually tracing through the source code to determine what the output would be if it was run.
Memorizing the source code and tracing through it in your head.
Constraining the input moves to ones where every Go program would make the same move, then using the output of one of a different Go program as your input.
Corollary: you can’t use any automation whatsoever to determine what move to make. Any automated system that can allow you to make accurate predictions is effectively a Go program.
Overall, then you might just want to prohibit the use of Turing machines. However, my understanding is that this results in a ban on algorithms. I don’t have enough CS to say what’s left to us if we’re denied algorithms.
Here’s a second operationalization of “know.” You’re allowed to train up using all the computerized help you want. But then, to prove your ability, you have to perfectly predict the output of the Go program on a set of randomly generated board positions, using only the power of your own brain. A softer criterion is to organize a competition, where participants are ranked by Brier score on this challenge.
However, this version of the success criterion is just a harder version of being an inhumanly good Go player. Not only do you have to play as well as the best Go program, you have to match its play. It’s the difference between being a basketball player with stats as good as Michael Jordan’s, and literally being able to copy his every move in novel situations indefinitely.
Neither of these versions of the success criterion operationalization seems particularly interesting. Both are too restrictive to be relevant to AI safety.
Did you have a different operationalization in mind?
I was thinking more of propositional knowledge (well, actually belief, but it doesn’t seem like this was a sticking point with anybody). A corollary of this is that you would be able to do this second operationalization, but possibly with the aid of a computer program that you wrote yourself that wasn’t just a copy of the original program. This constraint is slightly ambiguous but I think it shouldn’t be too problematic in practice.
The actual thing I had in mind was “come up with a satisfactory operationalization”.
I’m going to assume it’s impossible for me, personally, to outplay the best Go AI I have access to. Given that, the requirement is for me to write a better Go AI than the one I currently have access to.
Of course, that would mean that my new self-written program is now the best Go AI. So then I would be back to square one.
There are weaker computational machines than Turing machines, like regexes. But you don really care about that, you just want to ban automatic reasoning. I think it’s impossible to succeed with that constrain; Playing Go is hard, people can’t just read code that plays Go well and “learn from it.”