The supercomputers will be there whether we like it or not. Some of what they run will be attempts at AI. This is so far the only approach that someone unaware of Friendliness issues has a high probability of trying and succeeding with (and not immediately killing us all)
Numerous Un-augmented accelerated uplaods is a narrow safe path, and one we probably won’t follow, but it is a safe path. (so far one of 2, so it’s important) I think the likely win is less than FAI, but the dropoff isn’t so steep either as you walk off the path. Any safe AI approach will suggest profitable nonsafe alternatives.
An FAI failure is almost certainly alien, or not there yet. An augmentation failure is probably less-capable, probably not hostile, probably not strictly executing a utility function, and above all: can be surrounded by other, faster, uploads.
If the first pupload costs half a billion dollars and runs very slow, then even tweaking it will be safer than say, letting neural nets evolve in a rich environment on the same hardware.
The supercomputers will be there whether we like it or not. Some of what they run will be attempts at AI. This is so far the only approach that someone unaware of Friendliness issues has a high probability of trying and succeeding with (and not immediately killing us all)
Numerous Un-augmented accelerated uplaods is a narrow safe path, and one we probably won’t follow, but it is a safe path. (so far one of 2, so it’s important) I think the likely win is less than FAI, but the dropoff isn’t so steep either as you walk off the path. Any safe AI approach will suggest profitable nonsafe alternatives.
An FAI failure is almost certainly alien, or not there yet. An augmentation failure is probably less-capable, probably not hostile, probably not strictly executing a utility function, and above all: can be surrounded by other, faster, uploads.
If the first pupload costs half a billion dollars and runs very slow, then even tweaking it will be safer than say, letting neural nets evolve in a rich environment on the same hardware.