Aligned AGI is a large scale engineering task
Humans have never completed at large scale engineering task without at least one mistake
An AGI that has at least one mistake in its alignment model will be unaligned
Given enough time, an unaligned AGI will perform an action that will negatively impact human survival
Humans wish to survive
Therefore, humans ought not to make an AGI until one of the above premises changes.
This is another concise argument around AI x-risk. It is not perfect. What flaw in this argument do you consider the most important?
Current theme: default
Less Wrong (text)
Less Wrong (link)
Arrow keys: Next/previous image
Escape or click: Hide zoomed image
Space bar: Reset image size & position
Scroll to zoom in/out
(When zoomed in, drag to pan; double-click to close)
Keys shown in yellow (e.g., ]) are accesskeys, and require a browser-specific modifier key (or keys).
]
Keys shown in grey (e.g., ?) do not require any modifier keys.
?
Esc
h
f
a
m
v
c
r
q
t
u
o
,
.
/
s
n
e
;
Enter
[
\
k
i
l
=
-
0
′
1
2
3
4
5
6
7
8
9
→
↓
←
↑
Space
x
z
`
g
What are the flaws in this AGI argument?
Aligned AGI is a large scale engineering task
Humans have never completed at large scale engineering task without at least one mistake
An AGI that has at least one mistake in its alignment model will be unaligned
Given enough time, an unaligned AGI will perform an action that will negatively impact human survival
Humans wish to survive
Therefore, humans ought not to make an AGI until one of the above premises changes.
This is another concise argument around AI x-risk. It is not perfect. What flaw in this argument do you consider the most important?