Referring to belief propagation? The actual procedure does something really simple: ignore the problem. Experimentally, this approach has shown itself to be very good in a lot of cases. Very little is known about what determines how good an approximation this is, but if I recall correctly, it’s been proven that a single loop will always converge to the correct values; it’s also been proven that if all the local probability distributions are Gaussian, then the estimated means will also converge correctly, but the variances might not.
Many things can be done to improve the situation, too, but I’m not “up” on that at the moment.
From what I recall of reading Pearl (Probabilistic Reasoning in Intelligent Systems), there are a few other ways to do it.
One is to collapse the cycle down to one multi-input, multi-output node, so you’re conditioning on all inputs. This makes the node more complex, but forces you to consider the beliefs together, and prevent the information cascade problem.
Another is to make each message carry information about where it originated so that you can spot where evidence is being double-counted. (This is analogous to the “cite your sources” requirement in scholarship so that a faulty piece of evidence can be traced through multiple papers back to its source.)
Generally, to find ways around the problem, think back to the “soldier counting” problem and see what solutions would work there. The soldier counting problem is where you have some network of soldiers—ideally, without cycles—and want to know how you can count the size of the network by only looking at messages from soldiers connected to you. With an acyclic network, anyone can send the message “count” to all connections, and all soliders do the same until they’re connect to no one, at which point they return “1” plus the sum of any messages coming back.
Referring to belief propagation? The actual procedure does something really simple: ignore the problem. Experimentally, this approach has shown itself to be very good in a lot of cases. Very little is known about what determines how good an approximation this is, but if I recall correctly, it’s been proven that a single loop will always converge to the correct values; it’s also been proven that if all the local probability distributions are Gaussian, then the estimated means will also converge correctly, but the variances might not.
Many things can be done to improve the situation, too, but I’m not “up” on that at the moment.
From what I recall of reading Pearl (Probabilistic Reasoning in Intelligent Systems), there are a few other ways to do it.
One is to collapse the cycle down to one multi-input, multi-output node, so you’re conditioning on all inputs. This makes the node more complex, but forces you to consider the beliefs together, and prevent the information cascade problem.
Another is to make each message carry information about where it originated so that you can spot where evidence is being double-counted. (This is analogous to the “cite your sources” requirement in scholarship so that a faulty piece of evidence can be traced through multiple papers back to its source.)
Generally, to find ways around the problem, think back to the “soldier counting” problem and see what solutions would work there. The soldier counting problem is where you have some network of soldiers—ideally, without cycles—and want to know how you can count the size of the network by only looking at messages from soldiers connected to you. With an acyclic network, anyone can send the message “count” to all connections, and all soliders do the same until they’re connect to no one, at which point they return “1” plus the sum of any messages coming back.