If you think about how mere humans do things, we generate lots of tries, many/most of them dead ends or even dangers. We have to edit ourselves to get something good really good. But then biological evolution is like that, isn’t it?
I suppose that the dream of a super-intelligent AI is, among other things (perhaps), the dream of an engine that goes straight for the good stuff, never digressing, never making a false start, never even hinting at evil. I don’t believe it. Alignment is messy, and always will be. And resistance if futile.
If you think about how mere humans do things, we generate lots of tries, many/most of them dead ends or even dangers. We have to edit ourselves to get something good really good. But then biological evolution is like that, isn’t it?
I suppose that the dream of a super-intelligent AI is, among other things (perhaps), the dream of an engine that goes straight for the good stuff, never digressing, never making a false start, never even hinting at evil. I don’t believe it. Alignment is messy, and always will be. And resistance if futile.