People have tried lots and lots of approaches to getting good performance out of computers, including lots of “scary seeming” approaches
I won’t say I could predict that these wouldn’t foom ahead of time, but it seems the result of all of these is an AI engineer that is much much more narrow / less capable than a human AI researcher.
It makes me really scared, many people’s response to not getting mauled after poking a bear is to poke it some more. I wouldn’t care so much if I didn’t think the bear was going to maul me, my family, and everyone I care about.
I don’t expect a sudden jump where AIs go from being better at some tasks and worse at others, to being universally better at all tasks.
The relevant task for AIs to get better at is “engineering AIs that are good at performing tasks.” It seems like that task should have some effect on how quickly the AIs improve at that task, and others.
real-world data in high dimensions basically never look like spheres
This is a really good point. I would like to see a lot more research into the properties of mind space and how they affect generalization of values and behaviors across extreme changes in the environment, such as those that would be seen going from an approximately human level intelligence to a post foom intelligence.
The Security Mindset and Parenting: How to Provably Ensure your Children Have Exactly the Goals You Intend.
A good person is what you get when you raise a human baby in a good household, not what you get when you raise a computer program in a good household. Most people do not expect their children will grow up to become agents capable of out planning all other agents in the environment. If they did, I might appreciate if they read that book.
I won’t say I could predict that these wouldn’t foom ahead of time, but it seems the result of all of these is an AI engineer that is much much more narrow / less capable than a human AI researcher.
It makes me really scared, many people’s response to not getting mauled after poking a bear is to poke it some more. I wouldn’t care so much if I didn’t think the bear was going to maul me, my family, and everyone I care about.
The relevant task for AIs to get better at is “engineering AIs that are good at performing tasks.” It seems like that task should have some effect on how quickly the AIs improve at that task, and others.
This is a really good point. I would like to see a lot more research into the properties of mind space and how they affect generalization of values and behaviors across extreme changes in the environment, such as those that would be seen going from an approximately human level intelligence to a post foom intelligence.
A good person is what you get when you raise a human baby in a good household, not what you get when you raise a computer program in a good household. Most people do not expect their children will grow up to become agents capable of out planning all other agents in the environment. If they did, I might appreciate if they read that book.