In other words, control worse now, in order to...what?
Suppose the loss is mean-square deviation from the set point. Suppose there’s going to be a giant uncontrollable exogenous heat source soon (crowded party), and suppose there is no cooling system (the thermostat is hooked up to a heater but there is no AC).
Then we’re expecting a huge contribution to the loss function from an upcoming positive temperature deviation. And there’s nothing much the system can do about it once the party is going, other than obviously not turn on the heat and make it even worse.
But supposing the system knows this is going to happen, it can keep the room a bit too cool before the party starts. That also incurs a loss, of course. But the way mean-square-loss works is that we come out ahead on average.
Like, if the deviation is 0° now and then +10° midway through the party, that’s higher-loss than −2° now and +8° midway through the party, again assuming loss = mean-square-deviation. 0²+10² > 2²+8², right?
This sounds like a product of Sirius Cybernetics Corporation. “It is very easy to be blinded to the essential uselessness of them by the sense of achievement you get from getting them to work at all.”
All you need is a bimetallic strip and a pair of contacts to sense “too high” and “too low”.
Well jeez, I’m not proposing that we actually do this! I thought the “giant supercomputer cluster” was a dead giveaway.
If you want a realistic example, I do think the brain uses generative modeling / MPC as part of its homeostatic / allostatic control systems (and motor control and so on). I think there are good reasons that the brain does it that way, and that alternative model-free designs would not work as well (although they would work more than zero).
Suppose the loss is mean-square deviation from the set point. Suppose there’s going to be a giant uncontrollable exogenous heat source soon (crowded party), and suppose there is no cooling system (the thermostat is hooked up to a heater but there is no AC).
Then we’re expecting a huge contribution to the loss function from an upcoming positive temperature deviation. And there’s nothing much the system can do about it once the party is going, other than obviously not turn on the heat and make it even worse.
But supposing the system knows this is going to happen, it can keep the room a bit too cool before the party starts. That also incurs a loss, of course. But the way mean-square-loss works is that we come out ahead on average.
Like, if the deviation is 0° now and then +10° midway through the party, that’s higher-loss than −2° now and +8° midway through the party, again assuming loss = mean-square-deviation. 0²+10² > 2²+8², right?
Well jeez, I’m not proposing that we actually do this! I thought the “giant supercomputer cluster” was a dead giveaway.
If you want a realistic example, I do think the brain uses generative modeling / MPC as part of its homeostatic / allostatic control systems (and motor control and so on). I think there are good reasons that the brain does it that way, and that alternative model-free designs would not work as well (although they would work more than zero).