Yup! And yeah I think those are open research questions—inference over certain kinds of non-parametric Bayesian models is tractable, but not in general. What makes me optimistic is that humans in similar cultures have similar priors over vast spaces of goals, and seem to do inference over that vast space in a fairly tractable manner. I think things get harder when you can’t assume shared priors over goal structure or task structure, both for humans and machines.
Yup! And yeah I think those are open research questions—inference over certain kinds of non-parametric Bayesian models is tractable, but not in general. What makes me optimistic is that humans in similar cultures have similar priors over vast spaces of goals, and seem to do inference over that vast space in a fairly tractable manner. I think things get harder when you can’t assume shared priors over goal structure or task structure, both for humans and machines.