Are you saying that if an AI could be built with an explicitly programmed with a sense of identity and a self-preservation goal, then we could get a measure of safety by including humanity in its sense of identity ? That sounds rather indirect—why not include preservation of humanity in the goal ?
Or are you expecting that a sense of identity and self-preservation will arise as naturally, and thinking about ways of getting humanity into that sense of identity ?
Assuming that the AI drive concept of identity comes about when the AI notices where its goal system is written down (and instrumentally protects that place), then extraneous things like humanity aren’t going to be in that sense of identity. Unless it considers humanity as being where its goal is “written down” ? (Is that something like what Alicorn was suggesting ?)
Are you saying that if an AI could be built with an explicitly programmed with a sense of identity and a self-preservation goal, then we could get a measure of safety by including humanity in its sense of identity ? That sounds rather indirect—why not include preservation of humanity in the goal ?
Or are you expecting that a sense of identity and self-preservation will arise as naturally, and thinking about ways of getting humanity into that sense of identity ?
Assuming that the AI drive concept of identity comes about when the AI notices where its goal system is written down (and instrumentally protects that place), then extraneous things like humanity aren’t going to be in that sense of identity. Unless it considers humanity as being where its goal is “written down” ? (Is that something like what Alicorn was suggesting ?)