I have not studied control theory, but I think a PID controller may be the Bayes-optimal controller if:
the system is a second-order linear system with constant coefficients,
the system is controllable,
all disturbances in the system are additive white noise forcing terms,
there is no noise in perception,
the cost functional is the integral of the square of the error,
the time horizon is infinite in both directions (no transients), and
the prior belief distribution over possible reference signals is the same as if the reference signal was a Brownian motion (which needs first-order control) plus an integral of a Brownian motion (which needs second-order control).
What makes it a full Bayesian decision problem is the prior belief distribution over possible reference signals. At each time, you don’t know what the future reference signal is going to be, but you have a marginal posterior belief distribution over possible future reference signals given what the reference signal has been in the past. Part of this knowledge about possible future reference signals is represented in the state of the system you have been controlling, and part of it is represented in the state of the I element of the controller. You also don’t know what the delayed effects of past disturbances will be, but you have a marginal posterior belief distribution over possible future delayed effects given what the perception signal has been in the past. Part of this knowledge is also represented in the system and in the I element. (Not all of your knowledge about possible future reference signals and possible future delayed effects of past disturbances is represented, only your knowledge about possible future differences between them.) This representation is related to sufficient statistics (“sufficiency is the property possessed by a statistic . . . when no other statistic which can be calculated from the same sample provides any additional information”) and to updating of the parameter for a parametric family of belief distributions.
In a real engineering problem, the true belief about expected possible reference signals would be more specific than a belief of a random Brownian motion. But if a reference signal would not be improbable for Brownian motion, then a PID controller can still do well on that reference signal.
I think these conditions are sufficient but not necessary. If I knew control theory I would tell you more general conditions. If the cost functional has a term for the integral of the squared control signal, then a PID controller may not be optimal without added filters to keep the control signal from having infinite power.
Example 6.3-1 in Optimal Control and Estimation by Robert F. Stengel (1994 edition, pp. 540-541) is about PID controllers as optimal regulators in linear-quadratic-Gaussian control problems.
I see optimal control theory as the shared generalization of Bayesian decision networks and dynamic Bayesian networks in the continuous-time limit. (Dynamic Bayesian networks are Bayes nets which model how variables change over discretized time steps. When the time step size goes to zero and the variables are continuous, the limit is stochastic differential equations such as the equations of Brownian motion. When the time step size goes to zero and the variables are discrete, the limit is almostUri Nodelman’s continuous-time Bayesian networks. Bayesian decision networks are Bayes nets which represent a decision problem and contain decision nodes, utility nodes, and information arcs.)
(Not all of your knowledge about possible future reference signals and possible future delayed effects of past disturbances is represented, only your knowledge about possible future differences between them.)
So this isn’t a sufficient statistic, it’s only a sufficient-for-policy-implications statistic. Is there a name for that?
All “sufficient” statistics are only “sufficient” for some particular set of policy or epistemic implications. You could always care about the number of 1 bits, if you’re allowed to care about anything.
Then every “sufficient-for-policy-implications” statistic can become a “sufficient-for-implications-for-beliefs-about-the-future” statistic, under a coarsening of the sample space by some future-action-preserving and conditional-ratios-of-expected-payoff-differences-preserving equivalence relation?
(Would we expect deliberative thinking and memory to physically approximate such coarsenings, as linear controllers do?)
I have not studied control theory, but I think a PID controller may be the Bayes-optimal controller if:
the system is a second-order linear system with constant coefficients,
the system is controllable,
all disturbances in the system are additive white noise forcing terms,
there is no noise in perception,
the cost functional is the integral of the square of the error,
the time horizon is infinite in both directions (no transients), and
the prior belief distribution over possible reference signals is the same as if the reference signal was a Brownian motion (which needs first-order control) plus an integral of a Brownian motion (which needs second-order control).
What makes it a full Bayesian decision problem is the prior belief distribution over possible reference signals. At each time, you don’t know what the future reference signal is going to be, but you have a marginal posterior belief distribution over possible future reference signals given what the reference signal has been in the past. Part of this knowledge about possible future reference signals is represented in the state of the system you have been controlling, and part of it is represented in the state of the I element of the controller. You also don’t know what the delayed effects of past disturbances will be, but you have a marginal posterior belief distribution over possible future delayed effects given what the perception signal has been in the past. Part of this knowledge is also represented in the system and in the I element. (Not all of your knowledge about possible future reference signals and possible future delayed effects of past disturbances is represented, only your knowledge about possible future differences between them.) This representation is related to sufficient statistics (“sufficiency is the property possessed by a statistic . . . when no other statistic which can be calculated from the same sample provides any additional information”) and to updating of the parameter for a parametric family of belief distributions.
In a real engineering problem, the true belief about expected possible reference signals would be more specific than a belief of a random Brownian motion. But if a reference signal would not be improbable for Brownian motion, then a PID controller can still do well on that reference signal.
I think these conditions are sufficient but not necessary. If I knew control theory I would tell you more general conditions. If the cost functional has a term for the integral of the squared control signal, then a PID controller may not be optimal without added filters to keep the control signal from having infinite power.
Example 6.3-1 in Optimal Control and Estimation by Robert F. Stengel (1994 edition, pp. 540-541) is about PID controllers as optimal regulators in linear-quadratic-Gaussian control problems.
I see optimal control theory as the shared generalization of Bayesian decision networks and dynamic Bayesian networks in the continuous-time limit. (Dynamic Bayesian networks are Bayes nets which model how variables change over discretized time steps. When the time step size goes to zero and the variables are continuous, the limit is stochastic differential equations such as the equations of Brownian motion. When the time step size goes to zero and the variables are discrete, the limit is almost Uri Nodelman’s continuous-time Bayesian networks. Bayesian decision networks are Bayes nets which represent a decision problem and contain decision nodes, utility nodes, and information arcs.)
So this isn’t a sufficient statistic, it’s only a sufficient-for-policy-implications statistic. Is there a name for that?
All “sufficient” statistics are only “sufficient” for some particular set of policy or epistemic implications. You could always care about the number of 1 bits, if you’re allowed to care about anything.
Then every “sufficient-for-policy-implications” statistic can become a “sufficient-for-implications-for-beliefs-about-the-future” statistic, under a coarsening of the sample space by some future-action-preserving and conditional-ratios-of-expected-payoff-differences-preserving equivalence relation?
(Would we expect deliberative thinking and memory to physically approximate such coarsenings, as linear controllers do?)
Thank you for those references—exactly the sort of thing I’ve been looking for.