There is a difference between science and technology. Science is just empirical knowledge and the process of acquiring it. Technology is applied science. It shows hubris to actually think you can build something smarter than yourself in a way that has a reasonable chance of avoiding terrible unforeseen consequences.
That doesn’t mean he is wrong when he argues AGI is just on the horizon and it is better that someone who cares about friendliness implements it first. I don’t know enough to say one way or another.
Hubris is probably one of the traits that has propelled humanity to it’s current stage of civilization. It is debatable whether that was a good thing. Certainly not from a negative utilitarian perspective, but definitely from an aggregate utilitarian calculus.
Now that we are at this stage of human civilization, all we can do is continue onward. Humans taking their destiny into their own hands has the potential to solve the problems of previous attempts to do just that.
It isn’t really debatable that the stories of Faust and the Tower of Babel, etc have a point, it is just a matter of figuring out to what degree. The whole idea of FAI inherently gives this idea some credence, but it says that since someone is creating the tech it might as well be us since we care about Friendliness. Oh yea, and by the way you get Singularities and stuff if we don’t all die.
Tiiba,
There is a difference between science and technology. Science is just empirical knowledge and the process of acquiring it. Technology is applied science. It shows hubris to actually think you can build something smarter than yourself in a way that has a reasonable chance of avoiding terrible unforeseen consequences.
That doesn’t mean he is wrong when he argues AGI is just on the horizon and it is better that someone who cares about friendliness implements it first. I don’t know enough to say one way or another.
Hubris is probably one of the traits that has propelled humanity to it’s current stage of civilization. It is debatable whether that was a good thing. Certainly not from a negative utilitarian perspective, but definitely from an aggregate utilitarian calculus.
Now that we are at this stage of human civilization, all we can do is continue onward. Humans taking their destiny into their own hands has the potential to solve the problems of previous attempts to do just that.
It isn’t really debatable that the stories of Faust and the Tower of Babel, etc have a point, it is just a matter of figuring out to what degree. The whole idea of FAI inherently gives this idea some credence, but it says that since someone is creating the tech it might as well be us since we care about Friendliness. Oh yea, and by the way you get Singularities and stuff if we don’t all die.