Devaluing proposals only because they purportedly originated with an adversary.
In any technical discussion, there are a lot of well intentioned but otherwise not so well informed people participating.
On the part of individuals and groups both, this has the effect of creating protective layers of isolation and degrees of separation – between the qualified experts and everyone else.
While this is a natural tendency that can have a beneficial effect, the creation of too specific or strong of an ‘in-crowd’ can result in mono-culture effects.
The problem of a very poor signal to noise ratio from messages received from people outside of the established professional group basically means that the risk of discarding a good proposal from anyone regarded as an outsider is especially likely.
In terms of natural social process, there does not seem to be any available factor to counteract the possibility of forever increasing brittleness in the form of decreasing numbers of new ideas (ie; ‘echo chambers’).
Reactive devaluation: Bias in Evaluating AGI X-Risks
Devaluing proposals only because they purportedly originated with an adversary.
In any technical discussion, there are a lot of well intentioned but otherwise not so well informed people participating.
On the part of individuals and groups both, this has the effect of creating protective layers of isolation and degrees of separation – between the qualified experts and everyone else.
While this is a natural tendency that can have a beneficial effect, the creation of too specific or strong of an ‘in-crowd’ can result in mono-culture effects.
The problem of a very poor signal to noise ratio from messages received from people outside of the established professional group basically means that the risk of discarding a good proposal from anyone regarded as an outsider is especially likely.
In terms of natural social process, there does not seem to be any available factor to counteract the possibility of forever increasing brittleness in the form of decreasing numbers of new ideas (ie; ‘echo chambers’).
- link Wikipedia: Reactive devaluation
- an item on Forrest Landry’s compiled list of biases in evaluating extinction risks.