So the system needs to draw a distinction between just imagining freedom and making a plan for action that is predicted to actually produce freedom. This seems like something that a critic system can learn pretty easily. It’s known that the rodent dopamine system can learn blockers, such as not predicting reward when a blue light comes on at the same time as the otherwise reward-predictive red light.
There are 2 separable problems here: A. can a critic learn new abstract values?; B. how does the critic distinguish reality from imagination? I don’t see how blocking provides a realistic solution to either? Can you spell out what the blocker is and how it might solve these problems?
In general, these are both critical problems with the open-ended “super critic” hypothesis—how does Montague deal with these? So far, I don’t see any good solution except a strong grounding to basic survival-relevant values, as any crack in the system seems like it will quickly spiral out of control, much like heroin..
I’m a fan of Tomasello’s idea that social & sharing motivations provide the underlying fixed value function that drives most of human open-ended behavior. And there is solid evidence that humans vs. chimps differ strongly in these basic motivations, so it seems plausible that it is “built in”—curious to hear more about your doubts on that data?
In short, I strongly doubt that an open-ended critic is viable: it is just too easy to short-circuit (wirehead). The socially-grounded critic also has a strong potential for bad local minima: basically the “mutual admiration society” of self-reinforcing social currency. The result is cults of all forms, including that represented by one of the current major political parties in the US… But inevitably these are self-terminating when they conflict strongly with more basic survival values..
There are 2 separable problems here: A. can a critic learn new abstract values?; B. how does the critic distinguish reality from imagination? I don’t see how blocking provides a realistic solution to either? Can you spell out what the blocker is and how it might solve these problems?
In general, these are both critical problems with the open-ended “super critic” hypothesis—how does Montague deal with these? So far, I don’t see any good solution except a strong grounding to basic survival-relevant values, as any crack in the system seems like it will quickly spiral out of control, much like heroin..
I’m a fan of Tomasello’s idea that social & sharing motivations provide the underlying fixed value function that drives most of human open-ended behavior. And there is solid evidence that humans vs. chimps differ strongly in these basic motivations, so it seems plausible that it is “built in”—curious to hear more about your doubts on that data?
In short, I strongly doubt that an open-ended critic is viable: it is just too easy to short-circuit (wirehead). The socially-grounded critic also has a strong potential for bad local minima: basically the “mutual admiration society” of self-reinforcing social currency. The result is cults of all forms, including that represented by one of the current major political parties in the US… But inevitably these are self-terminating when they conflict strongly with more basic survival values..