I think the difference between direct benefit and net benefit is more general. That is, this post outlines a way in which accepting true observations that have already been collected might make the coalition net worse off at finding the truth, because of adversarial action in the external world. But when considering what observations to seek out in the future, the coalition faces tradeoffs (again, imposed by the world, but the practicalities of time and energy and other resources, rather than enemy action). If they document the differing kinds of beetles, that takes time and attention that could have been spent documenting the differing kinds of bees.
Whether beetles or bees are more interesting can of course only be estimated using the coalition’s current beliefs, rather than their beliefs after they have obtained the data they will obtain. The coalition might reasonably expect that time spent on beetles instead of bees is a ‘net waste’ even if the additional data on beetles is in fact expected to be useful, because of scarcity and expecting additional data on bees to be more useful.
It seems like there are lots of principles that make sense to apply here, so that they’re not just doing naive maximization, since this is a multi-armed bandit problem. They probably want to shrink estimates towards the mean, have an ensemble of models and have some of the budget directed by each individual model (and reallocate that budget afterwards), and various other things.
So it seems to me like you have to have this sort of ability to do cost-benefit tradeoffs given your current beliefs in order to operate at all in an embedded framework.
I think the difference between direct benefit and net benefit is more general. That is, this post outlines a way in which accepting true observations that have already been collected might make the coalition net worse off at finding the truth, because of adversarial action in the external world. But when considering what observations to seek out in the future, the coalition faces tradeoffs (again, imposed by the world, but the practicalities of time and energy and other resources, rather than enemy action). If they document the differing kinds of beetles, that takes time and attention that could have been spent documenting the differing kinds of bees.
Whether beetles or bees are more interesting can of course only be estimated using the coalition’s current beliefs, rather than their beliefs after they have obtained the data they will obtain. The coalition might reasonably expect that time spent on beetles instead of bees is a ‘net waste’ even if the additional data on beetles is in fact expected to be useful, because of scarcity and expecting additional data on bees to be more useful.
It seems like there are lots of principles that make sense to apply here, so that they’re not just doing naive maximization, since this is a multi-armed bandit problem. They probably want to shrink estimates towards the mean, have an ensemble of models and have some of the budget directed by each individual model (and reallocate that budget afterwards), and various other things.
So it seems to me like you have to have this sort of ability to do cost-benefit tradeoffs given your current beliefs in order to operate at all in an embedded framework.