This was extremely entertaining and also had good points. For now, just one question:
...The intern was arguing that minimizing prediction error would have all kinds of unintended bad effects. Which was crazy enough. The engineer was worse: they were arguing that Predict-O-Matic might maximize prediction error! Some kind of duality principle. Minimizing in one direction means maximizing in the other direction. Whatever that means.
Is this a reference to duality in optimization? If so, I don’t understand the formal connection?
No, it was a speculative conjecture which I thought of while writing.
The idea is that incentivizing agents to lower the error of your predictions (as in a prediction market) looks exactly like incentivizing them to “create” information (find ways of making the world more chaotic), and this is no coincidence. So perhaps there’s a more general principle behind it, where trying to incentivize minimization of f(x,y) only through channel x (eg, only by improving predictions) results in an incentive to maximize f through y, under some additional assumptions. Maybe there is a connection to optimization duality in there.
In terms of the fictional cannon, I think of it as the engineer trying to convince the boss by simplifying things and making wild but impressive sounding conjectures. :)
This was extremely entertaining and also had good points. For now, just one question:
Is this a reference to duality in optimization? If so, I don’t understand the formal connection?
No, it was a speculative conjecture which I thought of while writing.
The idea is that incentivizing agents to lower the error of your predictions (as in a prediction market) looks exactly like incentivizing them to “create” information (find ways of making the world more chaotic), and this is no coincidence. So perhaps there’s a more general principle behind it, where trying to incentivize minimization of f(x,y) only through channel x (eg, only by improving predictions) results in an incentive to maximize f through y, under some additional assumptions. Maybe there is a connection to optimization duality in there.
In terms of the fictional cannon, I think of it as the engineer trying to convince the boss by simplifying things and making wild but impressive sounding conjectures. :)