The demonstration that a fast and frugal satisficing algorithm won the competition defeats the widespread view that only “rational” algorithms can be accurate.
I am suspicious of work that attempts to provide evidence for a counterintuitive result in a way that could fairly obviously have been rigged. In this case, the key question is how “generic” their competition really was. It might be more convincing if arguments could be made about a plausible “real-world” distribution of problem instances, then a set of sample competitions drawn from that distribution and various decision algorithms run on those instances.
There is a lot more work on this point, not all of it focused on the point. What else, for example, would you call Robert Axelrod’s “Tit-for-Tat” than a “fast and frugal satisficing algorithm”? In fact, there has been enough other work on this and related points that I would not refer to as a counter-intuitive result.
Later work seems to support the notion of fast and frugal algorithms performing evenly with more complicated ones. See e.g. Fast and Frugal Heuristics: The Tools of Bounded Rationality for references to later experiments. (It’s unfortunately too late here for me to write a proper summary of it, especially since I haven’t read the referenced later studies.)
I am suspicious of work that attempts to provide evidence for a counterintuitive result in a way that could fairly obviously have been rigged. In this case, the key question is how “generic” their competition really was. It might be more convincing if arguments could be made about a plausible “real-world” distribution of problem instances, then a set of sample competitions drawn from that distribution and various decision algorithms run on those instances.
There is a lot more work on this point, not all of it focused on the point. What else, for example, would you call Robert Axelrod’s “Tit-for-Tat” than a “fast and frugal satisficing algorithm”? In fact, there has been enough other work on this and related points that I would not refer to as a counter-intuitive result.
Tit-for-tat doesn’t win because it’s computationally efficient.
Now that would be a cool extension of Axelrod’s test: include a penalty per round or per pairing as a function of algorithm length.
Length of source code, or running time? (How the heck did English end up with the same word for measuring time and a 3D axis?)
I had been thinking source code length, such that it would correspond to Kolmogorov complexity. Both would actually work, testing different things.
And perhaps the English question makes more sense if we consider things with a fourth time dimension ;)
Later work seems to support the notion of fast and frugal algorithms performing evenly with more complicated ones. See e.g. Fast and Frugal Heuristics: The Tools of Bounded Rationality for references to later experiments. (It’s unfortunately too late here for me to write a proper summary of it, especially since I haven’t read the referenced later studies.)
ok, sure. So there has been some thought put in beyond “here’s this one challenge that we rigged to make the fast and frugal algorithm look good!”