The difficulty of Friendliness is finite. The difficulties are big and subtle, but not unending.
How do we know that the problem is finite? When it comes to proving a computer program safe from being hacked the problem is considered NP-hard.
Google Chrome got recently hacked by chaining 14 different bugs together.
A working AGI is probably as least a complex as Google Chrome. Proving it safe will likely also be NP-hard.
How do we know that the problem is finite? When it comes to proving a computer program safe from being hacked the problem is considered NP-hard. Google Chrome got recently hacked by chaining 14 different bugs together. A working AGI is probably as least a complex as Google Chrome. Proving it safe will likely also be NP-hard.
Google Chrome doesn’t even self modify.