Does the first one being slow somehow make that other one less dangerous?
Yes, because I believe that the development will be gradually enough to tackle any risks on the way to a superhuman AGI, if superhuman capability is possible at all. There are certain limitations. Shortly after the invention of rocket science people landed on the moon. But the development eventually halted or slowed down. We haven’t reached other star systems yet. By that metaphor I want highlight that I am not aware of good arguments or other kinds of evidence indicating that an AGI would likely result in a run-away risk at any point of its development. It is possible but I am not sure that because of its low-probability we can reasonable neglect other existential risks. I believe that once we know how to create artificial intelligence capable of learning on a human level our comprehension of its associated risks and ability to limit its scope will have increased dramatically as well.
You’re using a different definition of AI than me. I’m thinking of ‘a mind running on a computer’ and you’re apparently thinking of ‘a human-like mind running on a computer’, where ‘human-like’ includes a lot of baggage about ‘what it means to be a mind’ or ‘what it takes to have a mind’.
I think any AI built from scratch will be a complete alien, and we won’t know just how alien until it starts doing stuff for reasons we’re incapable of understanding. And history has proven that the more sophisticated and complex the program, the more bugs, and the more it goes wrong in weird, subtle ways. Most such programs don’t have will, intent, or the ability to converse with you, making them substantially less likely to run away.
And again, you’re positing that people will understand, accept, and put limits in place, where there’s substantial incentives to let it run as free and as fast as possible.
Yes, because I believe that the development will be gradually enough to tackle any risks on the way to a superhuman AGI, if superhuman capability is possible at all. There are certain limitations. Shortly after the invention of rocket science people landed on the moon. But the development eventually halted or slowed down. We haven’t reached other star systems yet. By that metaphor I want highlight that I am not aware of good arguments or other kinds of evidence indicating that an AGI would likely result in a run-away risk at any point of its development. It is possible but I am not sure that because of its low-probability we can reasonable neglect other existential risks. I believe that once we know how to create artificial intelligence capable of learning on a human level our comprehension of its associated risks and ability to limit its scope will have increased dramatically as well.
You’re using a different definition of AI than me. I’m thinking of ‘a mind running on a computer’ and you’re apparently thinking of ‘a human-like mind running on a computer’, where ‘human-like’ includes a lot of baggage about ‘what it means to be a mind’ or ‘what it takes to have a mind’.
I think any AI built from scratch will be a complete alien, and we won’t know just how alien until it starts doing stuff for reasons we’re incapable of understanding. And history has proven that the more sophisticated and complex the program, the more bugs, and the more it goes wrong in weird, subtle ways. Most such programs don’t have will, intent, or the ability to converse with you, making them substantially less likely to run away.
And again, you’re positing that people will understand, accept, and put limits in place, where there’s substantial incentives to let it run as free and as fast as possible.
Sorry, I meant human-level learning capability when I said human like.