Thanks for the pointers to network science Jan, I don’t know this literature, and if it’s useful here then I’m glad you understand it well enough to guide us (and others) to key parts of it. I don’t see yet how to apply it to thinking quantitatively about scientific and forecasting communities.
If you (or another LWer) thinks that the theory around universality classes is applicable in thinking about how to ensure good info propagation in e.g. a scientific community, and you’re right, then I (and Jacob and likely many others) would love to read a summary, posted here as an answer. Might you explain how understanding the linked paper on universality classes has helped you think about info propagation in forecasting communities / related communities? Concrete heuristics would be especially interesting
(Note that Jacob and I have not taken a math course in topology or graph theory and won’t be able to read answers that assume such, though we’ve both studied formal fields of study and could likely pick it up quickly if it seemed practically useful.)
In general we’re not looking for *novel* contributions. To give an extreme example, if one person translates an existing theoretical literature into a fully fleshed out theory of info-cascades for scientific and forecasting communities, we’ll give them the entire prize pot.
Short summary of how is the lined paper important: you can think about bias as some sort of perturbation. You are then interested in the “cascade of spreading” of the perturbation, and especially factors like the distribution of sizes of cascades. The universality classes tell you this can be predicted by just a few parameters (Table 1 in the linked paper) depending mainly on local dynamic (forecaster-forecaster interactions). Now if you have a good model of the local dynamic, you can determine the parameters and determine into which universality class the problem belongs. Also you can try to infer the dynamics if you have good data on your interactions.
I’m afraid I don’t know enough about how “forecasting communities” work to be able to give you some good guesses what may be the points of leverage. One quick idea, if you have everybody on the same platform, may be to do some sort fo A/B experiment—manipulate the data so some forecasters would see the predictions of other with an artificially introduced perturbation, and see how their output will be different from the control group. If you have data on “individual dynamics” liken that, and some knowledge of network structure, the theory can help you predict the cascade size distribution.
(I also apologize for not being more helpful, but I really don’t have time to work on this for you.)
Thanks for the pointers to network science Jan, I don’t know this literature, and if it’s useful here then I’m glad you understand it well enough to guide us (and others) to key parts of it. I don’t see yet how to apply it to thinking quantitatively about scientific and forecasting communities.
If you (or another LWer) thinks that the theory around universality classes is applicable in thinking about how to ensure good info propagation in e.g. a scientific community, and you’re right, then I (and Jacob and likely many others) would love to read a summary, posted here as an answer. Might you explain how understanding the linked paper on universality classes has helped you think about info propagation in forecasting communities / related communities? Concrete heuristics would be especially interesting
(Note that Jacob and I have not taken a math course in topology or graph theory and won’t be able to read answers that assume such, though we’ve both studied formal fields of study and could likely pick it up quickly if it seemed practically useful.)
In general we’re not looking for *novel* contributions. To give an extreme example, if one person translates an existing theoretical literature into a fully fleshed out theory of info-cascades for scientific and forecasting communities, we’ll give them the entire prize pot.
Short summary of how is the lined paper important: you can think about bias as some sort of perturbation. You are then interested in the “cascade of spreading” of the perturbation, and especially factors like the distribution of sizes of cascades. The universality classes tell you this can be predicted by just a few parameters (Table 1 in the linked paper) depending mainly on local dynamic (forecaster-forecaster interactions). Now if you have a good model of the local dynamic, you can determine the parameters and determine into which universality class the problem belongs. Also you can try to infer the dynamics if you have good data on your interactions.
I’m afraid I don’t know enough about how “forecasting communities” work to be able to give you some good guesses what may be the points of leverage. One quick idea, if you have everybody on the same platform, may be to do some sort fo A/B experiment—manipulate the data so some forecasters would see the predictions of other with an artificially introduced perturbation, and see how their output will be different from the control group. If you have data on “individual dynamics” liken that, and some knowledge of network structure, the theory can help you predict the cascade size distribution.
(I also apologize for not being more helpful, but I really don’t have time to work on this for you.)