Reminds me of one of the early AI research projects using some variety of optimization algorithm to try to “learn” the ability to solve a wide variety of problems in a single program. Genetic algorithm I think, random mutation and cross-pollination of the programs between the best performers, that kind of thing.
After a while, they noticed that one of the lines that had developed, while not the best at any of the test problems, was second-best at all of them.
Yet when they tried to make it the base of all their next generation… it didn’t work...
Cue a massive analysis effort to rip it apart at the machine-code level and figure out what the heck was going on. Eventually they found that it had stumbled upon a security flaw in their running environment and learned to steal answers from the other programs running on the system.
Evolution doesn’t care if it’s cheating, only if it works.
Reminds me of one of the early AI research projects using some variety of optimization algorithm to try to “learn” the ability to solve a wide variety of problems in a single program. Genetic algorithm I think, random mutation and cross-pollination of the programs between the best performers, that kind of thing.
After a while, they noticed that one of the lines that had developed, while not the best at any of the test problems, was second-best at all of them.
Yet when they tried to make it the base of all their next generation… it didn’t work...
Cue a massive analysis effort to rip it apart at the machine-code level and figure out what the heck was going on. Eventually they found that it had stumbled upon a security flaw in their running environment and learned to steal answers from the other programs running on the system.
Evolution doesn’t care if it’s cheating, only if it works.