I feel like this metaphor doesn’t strike me as accurate because humanity can engage in commerce and insects cannot.
But also humanity causes a lot of environmental degradation but we still don’t actually want to bring about the wholesale destruction of the environment.
I have a few objections here:
Even when objectives aren’t aligned, that doesn’t mean the outcome is literally death. No corporation I interact with us aligned with me, but in many/most cases I am still better off for being able to transact with them.
I think there are plenty of scenarios where “humanity continues to exist” has benefits for AI—we are a source of training data and probably lots of other useful resources, and letting us continue to exist is not a huge investment, since we are mostly self-sustaining. Maybe this isn’t literally “being aligned” but I think supporting human life has instrumental benefits to AI.
I think the formal claim is only true inasmuch as it’s also true that the space of all objectives that align with the AI’s continued existence is also incredibly small. I think it’s much less clear how many of the objectives that are in some way supportive of the AI also result in human extinction.
This is a common problem with a lot of these hypothetical AI scenarios—WHY does the Oracle do this? How did the process of constructing this AI somehow make it want to eventually cause some negative consequence?
Next event will be April 30th—link.
Invite for the next meetup (March 24th) is now up—see here: https://www.lesswrong.com/events/Bf9rqXH93FtkAj6a5/twin-cities-acx-meetup-mar-2023
Invite for the next meetup (March 24th) is now up—see here: https://www.lesswrong.com/events/Bf9rqXH93FtkAj6a5/twin-cities-acx-meetup-mar-2023
Apologies for the late notice, the next event is this coming Saturday. RSVP here: https://www.lesswrong.com/events/jo6uxJuh4DLkx2Exx/twin-cities-acx-meetup-june-2023