Ah, OK, I buy that, thanks. What about the idea of building a system that doesn’t model itself or its past predictions, and asking it questions that don’t entail modeling any other superpredictors? (Like “what’s the likeliest way for a person to find the cure for Alzheimers, if we hypothetically lived in a world with no superpredictors or AGIs?”)
Ah, OK, I buy that, thanks. What about the idea of building a system that doesn’t model itself or its past predictions, and asking it questions that don’t entail modeling any other superpredictors? (Like “what’s the likeliest way for a person to find the cure for Alzheimers, if we hypothetically lived in a world with no superpredictors or AGIs?”)
Could work.