10-15 minutes isn’t much time. One thing I hope you touch on (perhaps in the ‘why it is hard’ portion) is the ambiguity - ‘friendly’ to whom or what?
friendly to me?
friendly to my values?
friendly to mankind?
friendly to mankind’s values?
It needs to be pointed out that these things are not necessarily all the same, and furthermore, they are probably not constant. People change. Populations change their members. An AI may be friendly to mankind when built, and be completely stable, but gradually become unfriendly to mankind because mankind changes (we changed quite a bit over the last million years, didn’t we?).
And for how hard the problem is, I’d emphasize how likely we are to be wrong about determining or extrapolating exactly what we want, in each sense listed above.
An engaging way to open a talk is to challenge the audience, then give them this info as first steps. If I can suggest an example, I’d say something like “Asimov’s 3 Laws looked innocuous and even in our best interest, but they were terribly flawed. So can we find a real solution? One that will hold for powerful AGI? This is an open problem.”
If you don’t run out of time, connect it back to self-modifying code (if only to ground the problem for a group of programmers). We need software that will self-modify to satisfy dynamic criteria that may exist, but are obscured to us. And we may only get one shot.
That will make someone ask about take-off and cooperating agents, and you can use the Q&A time to mention FOOMing, negentropy etc.
10-15 minutes isn’t much time. One thing I hope you touch on (perhaps in the ‘why it is hard’ portion) is the ambiguity - ‘friendly’ to whom or what?
friendly to me?
friendly to my values?
friendly to mankind?
friendly to mankind’s values?
It needs to be pointed out that these things are not necessarily all the same, and furthermore, they are probably not constant. People change. Populations change their members. An AI may be friendly to mankind when built, and be completely stable, but gradually become unfriendly to mankind because mankind changes (we changed quite a bit over the last million years, didn’t we?).
And for how hard the problem is, I’d emphasize how likely we are to be wrong about determining or extrapolating exactly what we want, in each sense listed above.
An engaging way to open a talk is to challenge the audience, then give them this info as first steps. If I can suggest an example, I’d say something like “Asimov’s 3 Laws looked innocuous and even in our best interest, but they were terribly flawed. So can we find a real solution? One that will hold for powerful AGI? This is an open problem.”
If you don’t run out of time, connect it back to self-modifying code (if only to ground the problem for a group of programmers). We need software that will self-modify to satisfy dynamic criteria that may exist, but are obscured to us. And we may only get one shot.
That will make someone ask about take-off and cooperating agents, and you can use the Q&A time to mention FOOMing, negentropy etc.