Most scary thing for me regarding AI is the simple fact that what constitutes modern human history is mostly lies repeated over and over, especially concentrated in the last 50+ years. With this in mind what exactly is AI being trained on and aligned to?
If it can’t discern truth from lies we are phucked.
If it can discern truth from lies we are phucked.
I understand the mechanisms of your metaphor and while they are undeniable they are also, IMO overly simplistic and not helpful as it leaves out the most important element of willpower.
Just like freewill, willpower implies making a decision then using your “freewill” or willpower to act on it. Willpower, as with freewill, is meaningless without making a conscious and preferably well informed decision.
Regardless of living or dead if you do not have reasonably accurate and somewhat complete info aka less wrong-ish its a crapshoot as to the outcome.
I posit that humans in general and Americans and most Europeans too have been so thoroughly and consistently lied to that any discussion of willpower or freewill is moot.
What IS being shut down or weeded out is truth, right and wrong, reality, critical thinking, honesty, and on and on, without which we all loose our ability to make good decisions.
I don’t know how our species can overcome this but IOM nothing else matters until we do.