Necro but maybe I can add something to the debate....
A problem I see is there are common cases where it is rational to be irrational, for example if being rational causes you emotional distress due to circumstances beyond your control.
And this is a big problem if one’s will to be “rational” is at root based on an emotional will to be “less wrong” for the purpose of improving internal feelings of one’s own value.
Because if that is the naked honest goal, then that rationalism is Hedonism by yet another name.
But realizing that might be destabilizing to the rationalist since, rationally, pure maximization of a social utilitarian value function is not a rational way to maximize a personal hedonistic value function no matter how hard one may try to contrive it....
So armed with intrepid rationality they may come to see the shades of grey rhetoric where Stalin is darker than Ghandi based on supposed utilitarian morality as a bad joke… and holding to utilitarian morality an irrational way to cope with the fact that power is everything and they have little of it.
To avoid this one would need to find a reason to be a rationalist other than their hedonistic value function. But the hedonistic value function is biological and innate, so the task is as impossible as winning the lottery.
But people do still win the lottery.
Would that suggest the difference between Stalin and Ghandi was little more than Ghandi’s bad luck? Because who really wouldn’t rather be in Stalin’s circumstances? (while of course believing they would avoid his evils and do good instead).
An uncomfortable thought… but then we’re always free to be irrational and just ignore what makes us uncomfortable...
I see the petition as a stunt or a mania depending on the participant. Because the danger of AI remains low to non-existent.
But as a way to get people excited about AI and more likely to overestimate the capabilities of the current AI it is very effective.
But the hurdles that remain remain as insurmountable as ever, such as, I don’t know… An AI that can actually understand anything.
Garbage in garbage out, a singularity will not make.