Hypothesis: power (status within military, government, academia, etc) is more obviously real to humans, and it takes a lot of work to build detailed, abstract models of anything other than this that feel as real. As a result people who have a basic understanding of a deep problem will consistently attempt to manoeuvre into powerful positions vaguely related to the problem, rather than directly solve the open problem. This will often get defended with “But even if we get a solution, how will we implement it?” without noticing that (a) there is no real effort by anyone else to solve the problem and (b) the more well-understood a problem is, the easier it is to implement a solution.
My, that was a long and difficult but worthwhile post. I see why you think it is not the natural state of affairs. Will think some more on it (though can’t promise a full response, it’s quite an effortful post). Am not sure I fully agree with your conclusions.
I’m much more interested in finding out what your model is after having tried to take those considerations into account, than I am in a point-by-point response.
(b) the more well-understood a problem is, the easier it is to implement a solution.
This might be true, but it doesn’t sound like it contradicts the premise of “how will we implement it”? Namely, just because understanding a problem makes it easier to implement, doesn’t mean that understanding alone makes it anywhere near easy to implement, and one may still need significant political clout in addition to having the solution. E.g. the whole infant nutrition thing.
It seems that AI safety has this issue less than every other problem in the world, by proportion of the people working on it.
Some double digit percentage of all of the people who are trying to improve the situation, are directly trying to solve the problem, I think? (Or maybe I just live in a bubble in a bubble.)
And I don’t know how well this analysis applies to non-AI safety fields.
I’d take a bet at even odds that it’s single-digit.
To clarify, I don’t think this is just about grabbing power in government or military. My outside view of plans to “get a PhD in AI (safety)” seems like this to me. This was part of the reason I declined an offer to do a neuroscience PhD with Oxford/DeepMind. I didn’t have any secret for why it might be plausibly crucial.
Hypothesis: power (status within military, government, academia, etc) is more obviously real to humans, and it takes a lot of work to build detailed, abstract models of anything other than this that feel as real. As a result people who have a basic understanding of a deep problem will consistently attempt to manoeuvre into powerful positions vaguely related to the problem, rather than directly solve the open problem. This will often get defended with “But even if we get a solution, how will we implement it?” without noticing that (a) there is no real effort by anyone else to solve the problem and (b) the more well-understood a problem is, the easier it is to implement a solution.
I think this is true for people who’ve been through a modern school system, but probably not a human universal.
My, that was a long and difficult but worthwhile post. I see why you think it is not the natural state of affairs. Will think some more on it (though can’t promise a full response, it’s quite an effortful post). Am not sure I fully agree with your conclusions.
I’m much more interested in finding out what your model is after having tried to take those considerations into account, than I am in a point-by-point response.
This seems like a good conversational move to have affordance for.
This might be true, but it doesn’t sound like it contradicts the premise of “how will we implement it”? Namely, just because understanding a problem makes it easier to implement, doesn’t mean that understanding alone makes it anywhere near easy to implement, and one may still need significant political clout in addition to having the solution. E.g. the whole infant nutrition thing.
Seems related to Causal vs Social Reality.
Do you have an example of a problem that gets approached this way?
Global warming? The need for prison reform? Factory Farming?
AI.
It seems that AI safety has this issue less than every other problem in the world, by proportion of the people working on it.
Some double digit percentage of all of the people who are trying to improve the situation, are directly trying to solve the problem, I think? (Or maybe I just live in a bubble in a bubble.)
And I don’t know how well this analysis applies to non-AI safety fields.
I’d take a bet at even odds that it’s single-digit.
To clarify, I don’t think this is just about grabbing power in government or military. My outside view of plans to “get a PhD in AI (safety)” seems like this to me. This was part of the reason I declined an offer to do a neuroscience PhD with Oxford/DeepMind. I didn’t have any secret for why it might be plausibly crucial.
Strong agree with Jacob.