You could have a system that automatically stopped when you reached a system with a certain level of tested intelligence. But perhaps I’m misunderstanding you.
With respect to the simulation argument, would you mind reminding us of why we would be overwhelmingly likely to be in a simulation if we build AI which ‘achieves takeoff acceleration’, if it’s quick to summarize? I expect many people haven’t read the simulation argument paper.
Having a system that stops when you reach a level of tested intelligence sounds appealing, but I’d be afraid of the measure of intelligence at hand being too fuzzy.
So the system would not detect it had achieved that level of intelligence, and it would bootstrap and take off in time to destroy the control system which was supposed to halt it. This would happen if we failed to solve any of many distinct problems that we don’t know how to solve yet, like symbol grounding, analogical and metaphorical processing and 3 more complex ones that I can think of, but don’t want to spend resources in this thread mentioning. Same for simulation argument.
You could have a system that automatically stopped when you reached a system with a certain level of tested intelligence. But perhaps I’m misunderstanding you.
With respect to the simulation argument, would you mind reminding us of why we would be overwhelmingly likely to be in a simulation if we build AI which ‘achieves takeoff acceleration’, if it’s quick to summarize? I expect many people haven’t read the simulation argument paper.
Having a system that stops when you reach a level of tested intelligence sounds appealing, but I’d be afraid of the measure of intelligence at hand being too fuzzy.
So the system would not detect it had achieved that level of intelligence, and it would bootstrap and take off in time to destroy the control system which was supposed to halt it. This would happen if we failed to solve any of many distinct problems that we don’t know how to solve yet, like symbol grounding, analogical and metaphorical processing and 3 more complex ones that I can think of, but don’t want to spend resources in this thread mentioning. Same for simulation argument.