My proposed experiment / test is trying to avoid analogizing humans, but rather scope out places where the ai can’t do very well. I’d like to avoid accidentally overly-narrow-scoping the vision of the tests. It won’t work with an ai network where the weights are reset every time.
An alternative, albeit massively-larger-scale experiment might be:
Will a self-driving car ever be able to navigate from one end of a city to another, using street signs and just learning the streets by exploring it?
A test of this might be like the following:
Randomly generate a simulated city/town, complete with street signs and traffic
Allow the self-driving car to peruse the city on its own accord
(or feed the ai network the map of the city a few times before the target destinations are given, if that is infeasible)
Give the self-driving car target destinations. Can the self-driving car navigate from one end of the city to the other, using only street signs, no GPS?
I think this kind of measuring would tell us how well our ai can handle open-endedness and help us understand where the void of progress is, and I think a small-scale chess experiment like this would help us shed light on bigger questions.
But humans play blindfold chess much slower than they read/write moves, they take tons of cognitive actions between each move. And at least when I play blindfold chess I need to lean heavily on my visual memory, and I often need to go back over the game so far for error-correction purposes, laboriously reading and writing to a mental scratchspace. I don’t know if better players do that.
I’m not sure why we shouldn’t expect an ai to be able to do well at it?
But an AI can do completely fine at the task by writing to an internal scratchspace. You are defining a restriction on what kind of AI is allowed, and I’m saying that human cognition probably doesn’t satisfy the analogous restrictions. I think to learn to play blindfold chess humans need to explicitly think about cognitive strategies, and the activity is much more similar to equipping an LM with the ability to write to its own context and then having it reason aloud about how to use that ability.
The reason why I don’t want a scratch-space, is because I view scratch space and context equivalent to giving the ai a notecard that it can peek at. I’m not against having extra categories or asterisks for the different kinds of ai for the small test.
Thinking aloud and giving it scratch space would mean it’s likely to be a lot more tractable for interpretability and alignment research, I’ll grant you that.
I appreciate the feedback, and I will think about your points more, though I’m not sure if I will agree.
My proposed experiment / test is trying to avoid analogizing humans, but rather scope out places where the ai can’t do very well. I’d like to avoid accidentally overly-narrow-scoping the vision of the tests. It won’t work with an ai network where the weights are reset every time.
An alternative, albeit massively-larger-scale experiment might be:
Will a self-driving car ever be able to navigate from one end of a city to another, using street signs and just learning the streets by exploring it?
A test of this might be like the following:
Randomly generate a simulated city/town, complete with street signs and traffic
Allow the self-driving car to peruse the city on its own accord
(or feed the ai network the map of the city a few times before the target destinations are given, if that is infeasible)
Give the self-driving car target destinations. Can the self-driving car navigate from one end of the city to the other, using only street signs, no GPS?
I think this kind of measuring would tell us how well our ai can handle open-endedness and help us understand where the void of progress is, and I think a small-scale chess experiment like this would help us shed light on bigger questions.
Just seems worth flagging that humans couldn’t do the chess test, and that there’s no particular reason to think that transformative AI could either.
I’m confused. What I’m referring to here is https://en.wikipedia.org/wiki/Blindfold_chess
I’m not sure why we shouldn’t expect an ai to be able to do well at it?
But humans play blindfold chess much slower than they read/write moves, they take tons of cognitive actions between each move. And at least when I play blindfold chess I need to lean heavily on my visual memory, and I often need to go back over the game so far for error-correction purposes, laboriously reading and writing to a mental scratchspace. I don’t know if better players do that.
But an AI can do completely fine at the task by writing to an internal scratchspace. You are defining a restriction on what kind of AI is allowed, and I’m saying that human cognition probably doesn’t satisfy the analogous restrictions. I think to learn to play blindfold chess humans need to explicitly think about cognitive strategies, and the activity is much more similar to equipping an LM with the ability to write to its own context and then having it reason aloud about how to use that ability.
The reason why I don’t want a scratch-space, is because I view scratch space and context equivalent to giving the ai a notecard that it can peek at. I’m not against having extra categories or asterisks for the different kinds of ai for the small test.
Thinking aloud and giving it scratch space would mean it’s likely to be a lot more tractable for interpretability and alignment research, I’ll grant you that.
I appreciate the feedback, and I will think about your points more, though I’m not sure if I will agree.