I plan on addressing false positives with a combination of sanity-checking/care-checking (“no, drinking tea probably doesn’t force me to sleep for exactly 6.5 hours the following night” or “so what if reading non-fiction makes me ravenous for spaghetti?”), and suggesting highest-information-content experimentation when neither of those applies (hopefully one would collect more data to test a hypothesis rather than immediately accept the program’s output in most cases). In this specific case, the raw conversation and bodily state data would probably not be nodes in the larger model—only the inferred “thing that really matters”, social life, would. Having constant feedback from the “expert”, who can choose which raw or derived variables to include in the model and which correlations don’t actually matter, seems to change the false positive problem.
I plan on addressing false positives with a combination of sanity-checking/care-checking (“no, drinking tea probably doesn’t force me to sleep for exactly 6.5 hours the following night” or “so what if reading non-fiction makes me ravenous for spaghetti?”), and suggesting highest-information-content experimentation when neither of those applies (hopefully one would collect more data to test a hypothesis rather than immediately accept the program’s output in most cases). In this specific case, the raw conversation and bodily state data would probably not be nodes in the larger model—only the inferred “thing that really matters”, social life, would. Having constant feedback from the “expert”, who can choose which raw or derived variables to include in the model and which correlations don’t actually matter, seems to change the false positive problem.