Eli, do you think you’re so close to developing a fully functional AGI that one more step and you might set off a land mine? Somehow I don’t believe you’re that close.
There is something else to consider. An AGI will ultimately be a piece of software. If you’re going to dedicate your life to talking about and ultimately writing a piece of software then you should have superb programming skills. You should code something.. anything.. just to learn to code. Your brain needs to swim in code. Even if none of that code ends up being useful the skill you gain will be. I have no doubt that you’re a good philosopher and a good writer since I have read your blog but wether or not you’re a good hacker is a complete mystery to me.
What’s the point of despair? There seems to be a given assumption in the original post that:
1) there is no protection, universe is allowed to be horrible --> 2)lets despair
But number 2 doesn’t change 1 one bit. This is not a clever argument to disprove number 1. I’m just saying despair is pointless if it changes nothing. It’s like when babies cry automatically when something isn’t the way they like because they are programmed to by evolution because this reliably attracted the attention of adults. Despairing about the universe will not attract the attention of adults to make it better. We are the only adults, that’s it. I would rather reason along the lines of:
1) there is no protection, universe is allowed to be horrible --> 2)what can I do to make it better
Agreed with everything else except the part where this is really sad news that’s supposed to make us unhappy.