Looks interesting. Thanks for doing this. It would be useful to me to get links to some of the things you mention (like eric drexler’s work, I am not familiar with).
I think there might be some categories missed here, it only presents the AI building problems. There are further classes of problem such as social problems. For example the problem of all agreeing not to develop AI until the other problems are solved.
There are also questions around whether we can make effective agents at all. We have existence proof on effective non-agent intelligences, but none around effective agent intelligences.
We invented the model of rational agents for humans, then we had to add lots and lots of exceptions (heuristics and biases). So much that you have disavowed it in this post. Yet we keep with the same model for AIs. Perhaps we need a kuhnian paradigm shift in our understanding of agents.
Looks interesting. Thanks for doing this. It would be useful to me to get links to some of the things you mention (like eric drexler’s work, I am not familiar with).
I think there might be some categories missed here, it only presents the AI building problems. There are further classes of problem such as social problems. For example the problem of all agreeing not to develop AI until the other problems are solved.
There are also questions around whether we can make effective agents at all. We have existence proof on effective non-agent intelligences, but none around effective agent intelligences.
We invented the model of rational agents for humans, then we had to add lots and lots of exceptions (heuristics and biases). So much that you have disavowed it in this post. Yet we keep with the same model for AIs. Perhaps we need a kuhnian paradigm shift in our understanding of agents.