It’s “vaguely magical” in sense that there is a large gap between what we have now and (U)FAI. We have no clear idea of how that gap could be crossed, we just wave hands and say “and then magic happens and we arrive at our destination”.
Many things are far beyond our current abilities, such as interstellar space travel. We have no clear idea of how humanity will travel to the stars, but the subject is neither “vaguely magical”, nor is it true that the sentence “humans will visit the stars” does not refer to anything.
I feel that it is an unfair characterization of the people who investigate AI risk to say that they claim it will happen by magic, and that they stop the investigation there. You could argue that their investigation is poor, but it is clear that they have worked a lot to investigate the processes that could lead to Unfriendly AI.
We have no clear idea of how humanity will travel to the stars, but the subject is neither “vaguely magical”, nor is it true that the sentence “humans will visit the stars” does not refer to anything.
We have no clear idea if or how humanity will travel to the stars. I feel that discussions of things like interstellar starship engines at the moment are “vaguely magical” since no known technology suffices and it’s not a “merely engineering” problem. Do you think it’s useful to work on safety of interstellar engines? They could blow up and destroy a whole potential colony…
You bring up a good point, whether it is useful to worry about UFAI.
To recap, my original query was about the claim that p(UFAI before 2116) is less than 1% due to UFAI being “vaguely magical”. I am interested in figuring out what that means—is it a fair representation of the concept to say that p(Interstellar before 2116) is less than 1% because interstellar travel is “vaguely magical”?
What would be the relationship between “Requiring Advanced Technology” and “Vaguely Magical”? Clarke’s third law is a straightforward link, but “vaguely magical” has previously been used to indicate poor definitions, poor abstractions and sentences that do not refer to anything.
I am not sure the OP had much meaning behind his “vaguely magical” expression, but given that we are discussing it anyway :-) I would probably reinterpret it in terms of Knightian uncertainty. It’s not only the case that we don’t know, we don’t know what we don’t know and how much we don’t know.
This interpretation makes a lot of sense. The term can describe events that have a lot of Knightian Uncertainty, which a “Black Swan” like UFAI certainly has.
It’s “vaguely magical” in sense that there is a large gap between what we have now and (U)FAI. We have no clear idea of how that gap could be crossed, we just wave hands and say “and then magic happens and we arrive at our destination”.
Many things are far beyond our current abilities, such as interstellar space travel. We have no clear idea of how humanity will travel to the stars, but the subject is neither “vaguely magical”, nor is it true that the sentence “humans will visit the stars” does not refer to anything.
I feel that it is an unfair characterization of the people who investigate AI risk to say that they claim it will happen by magic, and that they stop the investigation there. You could argue that their investigation is poor, but it is clear that they have worked a lot to investigate the processes that could lead to Unfriendly AI.
We have no clear idea if or how humanity will travel to the stars. I feel that discussions of things like interstellar starship engines at the moment are “vaguely magical” since no known technology suffices and it’s not a “merely engineering” problem. Do you think it’s useful to work on safety of interstellar engines? They could blow up and destroy a whole potential colony…
You bring up a good point, whether it is useful to worry about UFAI.
To recap, my original query was about the claim that p(UFAI before 2116) is less than 1% due to UFAI being “vaguely magical”. I am interested in figuring out what that means—is it a fair representation of the concept to say that p(Interstellar before 2116) is less than 1% because interstellar travel is “vaguely magical”?
What would be the relationship between “Requiring Advanced Technology” and “Vaguely Magical”? Clarke’s third law is a straightforward link, but “vaguely magical” has previously been used to indicate poor definitions, poor abstractions and sentences that do not refer to anything.
I am not sure the OP had much meaning behind his “vaguely magical” expression, but given that we are discussing it anyway :-) I would probably reinterpret it in terms of Knightian uncertainty. It’s not only the case that we don’t know, we don’t know what we don’t know and how much we don’t know.
This interpretation makes a lot of sense. The term can describe events that have a lot of Knightian Uncertainty, which a “Black Swan” like UFAI certainly has.