we should be all the more skeptical of our far beliefs, which are the most susceptible to bias.
Just out of interest… assume my far beliefs take the form of a probability distribution of possible future outcomes. How can I be “skeptical” of that? Given that something will happen in the future, all I can do is update in the direction of a different probability distribution.
In other words, which direction am I likely to be biased in?
We should update away from beliefs that the future will resemble a story, particularly a story whose primary danger will be fought by superheroes (most particularly for those of us who would personally be among the superheroes!) and towards beliefs that the future will resemble the past and the primary dangers will be drearily mundane.
The future will certainly resemble a story—or, more accurately, will be capable of being placed into several plausible narrative frames, just as the past has. The bias you’re probably trying to point to is in interpreting any particular plausible story as evidence for its individual components—or, for that matter, against.
The conjunction fallacy
implies that any particular vision of a Singularity-like outcome is less likely than our untrained intuitions would lead us to believe. It’s an excellent reason to be skeptical of any highly derived theories of the future—the specifics of Ray Kurzweil’s singularity timeline, for example, or Robin Hanson’s Malthusian emverse. But I don’t think it’s a good reason to be skeptical of any of the dominant singularity models in general form. Those don’t work back from a compelling image to first principles; most of them don’t even present specific consequential predictions, for fairly straightforward reasons. All the complexity is right there on the surface, and attempts to narrativize it inevitably run up against limits of imagination. (As evidence, the strong Singularity has been fairly poor at producing fiction when compared to most future histories of comparable generality; there’s no equivalent of Heinlein writing stories about nuclear-powered space colonization, although there’s quite a volume of stories about weak or partial singularities.)
So yes, there’s not going to be a singleton AI bent on turning us all into paperclips. But that’s a deliberately absurd instantiation of a much more general pattern. I can conceive of a number of ways in which the general pattern too might be wrong, but the conjunction fallacy doesn’t fly; a number of attempted debunkings, meanwhile, do suffer from narrative fixation issues.
Superhero bias is a more interesting question—but it’s also a more specific one.
Well, any sequence of events can be placed in a narrative frame with enough of a stretch, but the fact remains that different sequence of events differ in their amenability to this; fiction is not a random sampling from the space of possible things we could imagine happening, and the Singularity is narratively far stronger than most imaginable futures, to a degree that indicates bias we should correct for. I’ve seen a fair bit of strong Singularity fiction at this stage, though being, well, singular, it tends not to be amenable to repeated stories by the same author the way Heinlein’s vision of nuclear powered space colonization was.
Just out of interest… assume my far beliefs take the form of a probability distribution of possible future outcomes. How can I be “skeptical” of that? Given that something will happen in the future, all I can do is update in the direction of a different probability distribution.
In other words, which direction am I likely to be biased in?
In the direction of overconfidence, i.e., assigning too much probability mass to your highest probability theory.
We should update away from beliefs that the future will resemble a story, particularly a story whose primary danger will be fought by superheroes (most particularly for those of us who would personally be among the superheroes!) and towards beliefs that the future will resemble the past and the primary dangers will be drearily mundane.
The future will certainly resemble a story—or, more accurately, will be capable of being placed into several plausible narrative frames, just as the past has. The bias you’re probably trying to point to is in interpreting any particular plausible story as evidence for its individual components—or, for that matter, against.
The conjunction fallacy implies that any particular vision of a Singularity-like outcome is less likely than our untrained intuitions would lead us to believe. It’s an excellent reason to be skeptical of any highly derived theories of the future—the specifics of Ray Kurzweil’s singularity timeline, for example, or Robin Hanson’s Malthusian emverse. But I don’t think it’s a good reason to be skeptical of any of the dominant singularity models in general form. Those don’t work back from a compelling image to first principles; most of them don’t even present specific consequential predictions, for fairly straightforward reasons. All the complexity is right there on the surface, and attempts to narrativize it inevitably run up against limits of imagination. (As evidence, the strong Singularity has been fairly poor at producing fiction when compared to most future histories of comparable generality; there’s no equivalent of Heinlein writing stories about nuclear-powered space colonization, although there’s quite a volume of stories about weak or partial singularities.)
So yes, there’s not going to be a singleton AI bent on turning us all into paperclips. But that’s a deliberately absurd instantiation of a much more general pattern. I can conceive of a number of ways in which the general pattern too might be wrong, but the conjunction fallacy doesn’t fly; a number of attempted debunkings, meanwhile, do suffer from narrative fixation issues.
Superhero bias is a more interesting question—but it’s also a more specific one.
Well, any sequence of events can be placed in a narrative frame with enough of a stretch, but the fact remains that different sequence of events differ in their amenability to this; fiction is not a random sampling from the space of possible things we could imagine happening, and the Singularity is narratively far stronger than most imaginable futures, to a degree that indicates bias we should correct for. I’ve seen a fair bit of strong Singularity fiction at this stage, though being, well, singular, it tends not to be amenable to repeated stories by the same author the way Heinlein’s vision of nuclear powered space colonization was.