Here’s an inelegant answer that might work as a good proxy:
Hand-code a list of substructures that you identify as “good:” Gliders, glider guns, maybe some interesting shapes, etc. Make this list at least 10 items long.
Score is the product of the total numberXduration of each type of substructure. Use a large but finite grid, with a large but finite time.
I think this answer is better than it sounds. I think that value is complex; flourishing is not something that should be possible to specify simply in the real world, so why should it be possible in GoL? Moreover, flourishing in the real world involves a combination of order and disorder: Macro-structures like people and societies and spaceships (order) but they have to be different from each other and combined and recombined in lots of new ways (disorder).
I think one criticism of this answer is that it too could potentially be “gamed” by some big grid of densely packed substructures from the list. However, in practice I don’t think this will be a problem because I doubt it will be possible to construct such a grid. Nevertheless I think thinking more about this question for an hour would produce a better answer.
If you want to be able to track “can we create a variety of good structures, plus avoid bad structures?” you could weight the structures differently, and label some “bad”.
(I’m not sure this actually accomplishes the underlying philosophical goal, but seemed like the obvious extension from your current line of thinking)
Interesting thoughts, thanks. My concerns: 1) Diversity would be restricted to what I specify as interesting shapes, while perhaps what I really want is for the AI to be able to discover new ways to accomplish some target value. 2) From a technological perspective, may be too expensive to implement? (in that, at every pass, must search over all subsets of space and check against all (suitably-sized) patterns in the database in order to determine what reward to provide).
Both good points. I think the AI will find new ways to accomplish your value, for pretty much anything you set as your value, including this one. (I for one have very few ideas how the AI would manage to build all those shapes; wouldn’t they collide with each other? Probably some structure or organization would be needed. Etc.)
I don’t have good intuitions for what is easy or hard. Instead of checking all sub-regions at all times, you could randomly sample some sub-regions at some times; that would drastically reduce the expense while incentivizing the same behavior.
I notice something like “glider guns feel more valuable to me than gliders, and glider-gun-guns (is that a thing?) feel more valuable than glider-guns, but I wouldn’t want the universe tiled with either of them.”
I agree. I think I don’t want the universe tiled with anything really; one (or a handful) of tiles is enough. Maybe a step in the right direction:
Find a library of interesting GoL structures. Declare some to be bad and the rest good. Score = the number of good types that are instantiated minus the number of bad. So making more than one of the same structure doesn’t help. A more sophisticated algorithm would give you points equal to the log of the number of a structure.
Thinking about the question from another angle: What would I want the RL agent to end up doing? What would make me most pleased? Answer: If it constructed some sort of infinitely growing pattern, that spirals out from the middle filling the void with interesting oscillating structures of infinite diversity, such that eventually all stable structures of all sizes will be created. Even better would be if not all, but most, stable structures would be created—and some category of “bad” ones would never be.
Here’s an inelegant answer that might work as a good proxy:
Hand-code a list of substructures that you identify as “good:” Gliders, glider guns, maybe some interesting shapes, etc. Make this list at least 10 items long.
Score is the product of the total numberXduration of each type of substructure. Use a large but finite grid, with a large but finite time.
I think this answer is better than it sounds. I think that value is complex; flourishing is not something that should be possible to specify simply in the real world, so why should it be possible in GoL? Moreover, flourishing in the real world involves a combination of order and disorder: Macro-structures like people and societies and spaceships (order) but they have to be different from each other and combined and recombined in lots of new ways (disorder).
I think one criticism of this answer is that it too could potentially be “gamed” by some big grid of densely packed substructures from the list. However, in practice I don’t think this will be a problem because I doubt it will be possible to construct such a grid. Nevertheless I think thinking more about this question for an hour would produce a better answer.
If you want to be able to track “can we create a variety of good structures, plus avoid bad structures?” you could weight the structures differently, and label some “bad”.
(I’m not sure this actually accomplishes the underlying philosophical goal, but seemed like the obvious extension from your current line of thinking)
Yep, that seems like an improvement over the original version.
Interesting thoughts, thanks. My concerns: 1) Diversity would be restricted to what I specify as interesting shapes, while perhaps what I really want is for the AI to be able to discover new ways to accomplish some target value. 2) From a technological perspective, may be too expensive to implement? (in that, at every pass, must search over all subsets of space and check against all (suitably-sized) patterns in the database in order to determine what reward to provide).
Both good points. I think the AI will find new ways to accomplish your value, for pretty much anything you set as your value, including this one. (I for one have very few ideas how the AI would manage to build all those shapes; wouldn’t they collide with each other? Probably some structure or organization would be needed. Etc.)
I don’t have good intuitions for what is easy or hard. Instead of checking all sub-regions at all times, you could randomly sample some sub-regions at some times; that would drastically reduce the expense while incentivizing the same behavior.
I notice something like “glider guns feel more valuable to me than gliders, and glider-gun-guns (is that a thing?) feel more valuable than glider-guns, but I wouldn’t want the universe tiled with either of them.”
I agree. I think I don’t want the universe tiled with anything really; one (or a handful) of tiles is enough. Maybe a step in the right direction:
Find a library of interesting GoL structures. Declare some to be bad and the rest good. Score = the number of good types that are instantiated minus the number of bad. So making more than one of the same structure doesn’t help. A more sophisticated algorithm would give you points equal to the log of the number of a structure.
Thinking about the question from another angle: What would I want the RL agent to end up doing? What would make me most pleased? Answer: If it constructed some sort of infinitely growing pattern, that spirals out from the middle filling the void with interesting oscillating structures of infinite diversity, such that eventually all stable structures of all sizes will be created. Even better would be if not all, but most, stable structures would be created—and some category of “bad” ones would never be.