Anders_H

• Anders_HJul 4, 2022, 11:11 AM

  2 points

  in reply to: Big Steve’s comment on: The New Rid­dle of In­duc­tion: Neu­tral and Rel­a­tive Per­spec­tives on Color

  I don’t think the existence of lawlike phenomena is controversial, at least not on this forum. Otherwise, how do you account for the remarkable patterns to our observations? Of course, it is not possible to determine what those phenomena are, but I don’t think my solution requires this. It just requires that our sensory algorithm responds the same way every time.

• Anders_HJun 21, 2021, 2:20 PM

  2 points

  on: Why did no LessWrong dis­course on gain of func­tion re­search de­velop in 2013/​2014?

  I found the original website for Prof. Lipsitch’s “Cambridge Working Group” from 2014 at http://​​www.cambridgeworkinggroup.org/​​ . While the website does not focus exclusively on gain-of-function, this was certainly a recurring theme in his public talks about this.

  The list of signatories (which I believe has not been updated since 2016) includes several members of our community (apologies to anyone who I have missed):

  • Toby Ord, Oxford University

  • Sean O hEigeartaigh, University of Oxford

  • Daniel Dewey, University of Oxford

  • Anders Sandberg, Oxford University

  • Anders Huitfeldt, Harvard T.H. Chan School of Public Health

  • Viktoriya Krakovna, Harvard University PhD student

  • Dr. Roman V. Yampolskiy, University of Louisville

  • David Manheim, 1DaySooner

  Interestingly, there was an opposing group arguing in favor of this kind of research, at http://​​www.scientistsforscience.org/​​. I do not recognize a single name on their list of signatories

• Anders_HJun 19, 2021, 1:31 PM

  2 points

  in reply to: Anders_H’s comment on: Why did no LessWrong dis­course on gain of func­tion re­search de­velop in 2013/​2014?

  Here is a video of Prof. Lipsitch at EA Global Boston in 2017. I haven’t watched it yet, but I would expect him to discuss gain-of-function research: https://​​forum.effectivealtruism.org/​​posts/​​oKwg3Zs5DPDFXvSKC/​​marc-lipsitch-preventing-catastrophic-risks-by-mitigating

• Anders_HJun 19, 2021, 1:24 PM

  16 points

  on: Why did no LessWrong dis­course on gain of func­tion re­search de­velop in 2013/​2014?

  Here is a data point not directly relevant to Less Wrong, but perhaps to the broader rationality community:
  
  Around this time, Marc Lipsitch organized a website and an open letter warning publicly about the dangers of gain-of-function research. I was a doctoral student at HSPH at the time, and shared this information with a few rationalist-aligned organizations. I remember making an offer to introduce them to Prof. Lipsitch, so that maybe he could give a talk. I got the impression that the Future of Life Institute had some communication with him, and I see from their 2015 newsletter that there is some discussion of his work, but I am not sure if anything more concrete came out of of this
  
  My impression was that while they considered this important, this was more of a catastrophic risk than an existential risk, and therefore outside their core mission.

• Anders_HJun 15, 2021, 12:25 PM

  2 points

  in reply to: JenniferRM’s comment on: Shall we count the liv­ing or the dead?

  This comment touches on the central tension between the current paradigm in medicine, i.e. “evidence-based medicine” and an alternative and intuitively appealing approach based on a biological understanding of mechanism of disease.

  In evidence-based medicine, decisions are based on statistical analysis of randomized trials; what matters is whether we can be confident that the medication probabilistically has improved outcomes when tested on humans as a unit. We don’t care really care too much about the mechanism behind the causal effect, just whether we can be sure it is real.

  The exaggerated strawman alternative approach to EBM would be Star Trek medicine, where the ship’s doctor can reliably scan an alien’s biology, determine which molecule is needed to correct the pathology, synthesize that molecule and administer it as treatment.

  If we have a complete understanding of what Nancy Cartwright calls “the nomological machine”, Star Trek medicine should work in theory. However, you are going to need a very complete, accurate and detailed map of the human body to make it work. Given the complexity of the human body, I think we are very far from being able to do this in practice.

  There have been many cases in recent history where doctors believed they understood biology well enough to predict the consequences, yet were proved wrong by randomized trials. See for example Vinay Prasad’s book “Ending Medical Reversal”.

  My personal view is that we are very far from being able to ground clinical decisions in mechanistic knowledge instead of randomized trials. Trying to do so would probably be dangerous given the current state of biological understanding. However, we can probably improve on naive evidence-based medicine by carving out a role for mechanistic knowledge to complement data analysis. Mechanisms seems particularly important for reasoning correctly about extrapolation, the purpose of my research program is to clarify one way such mechanisms can be used. It doesn’t always work perfectly, but I am not aware of any examples where an alternative approach works better.

• Anders_HJun 15, 2021, 10:35 AM

  4 points

  in reply to: JenniferRM’s comment on: Shall we count the liv­ing or the dead?

  Thank you so much for writing this! Yes, this is mostly an accurate summary of my views (although I would certainly phrase some things differently). I just want to point out two minor disagreements:

  1. I don’t think the problem is that doctors are too rushed to do a proper job, I think the patient-specific data that you would need is in many cases theoretically unobservable, or at least that we would need a much more complete understanding of biological mechanisms in order to know what to test the patients for in order to make a truly individualized decision. At least for the foreseeable future, I think it will be impossible for doctors to determine which patients will benefit on an individual level, they will be constrained to using the patient’s observables to put them in a reference group, and then use that reference group to predict risk based on observations from other patients in the same reference group

  2. I am not entirely convinced that the Pearlian approach is the most natural way to handle this. In the manuscript, I use “modern causal models” as a more general term that also includes other types of counterfactual causal models. Of course, all these models are basically isomorphic, and Cinelli/​Pearl did show in response to my last paper that it is possible to do the same thing using DAGs. I am just not at all convinced that the easiest way to capture the relevant intuition is to use the Pearl’s graphical representation of the causal models.

• Anders_HJun 15, 2021, 10:24 AM

  2 points

  in reply to: JBlack’s comment on: Shall we count the liv­ing or the dead?

  You are correct that someone who has one allergy may be more likely to have an other allergy, and that this violates the assumptions of our model. Our model relies on a strong independence assumption, there are many realistic cases where this independence assumption will not hold. I also agree that the video uses an example where the assumption may not hold. The video is oversimplified on purpose, in an attempt to get people interested enough to read the arXiv preprint.

  If there is a small correlation between baseline risk and effect of treatment, this will have a negligible impact on the analysis. If there is a moderate correlation, you will probably be able to bound the true treatment effect using partial identification methods. If there is strong correlation, this may invalidate the analysis completely.
  
  The point we are making is not that the model will always hold exactly. Any model is an approximation. Let’s suppose we have three choices:

  1. Use a template for a causal model that “counts the living”, think about all the possible biological reasons that this model could go wrong, represent them in the model if possible, and account for them as best you can in the analysis

  2. Use a template for a causal model that “counts the dead”, think about all the possible biological reasons that this model could go wrong, represent them in the model if possible, and account for them as best you can in the analysis

  3. Use a model that is invariant to whether you count the living or the dead. This cannot be based on a multiplicative (relative risk) parameter.

  The third approach will not be sensitive to the particular problems that I am discussing, but all the suggested methods of this type have their own problems. I have written this earlier, my view is that these problems are more troubling than the problems with the relative risk models.

  What we are arguing in this preprint, is that if you decide to go with a relative risk model, you should choose between (1) and (2) based on the principles suggested by Sheps, and then reason about problems with this model and how it can be addressed in the analysis, based on the principles that you have correctly outlined in your comment.

  I can assure you that if you decide to go with a multiplicative model but choose the wrong “base case”, then all of the problems you have discussed in your comments will be orders of magnitude more difficult to deal with in any meaningful way. In other words, it is only after you make the choice recommended by Sheps that it even becomes possibly the meaningfully analyze the reasons for deviation from effect homogeneity...

• Anders_HJun 14, 2021, 8:18 PM

  2 points

  in reply to: George3d6’s comment on: Shall we count the liv­ing or the dead?

  I very emphatically disagree with this.

  You are right that once you have a prediction for risk if untreated, and a prediction risk if treated, you just need a cost/​benefit analysis. However, you won’t get to that stage without a paradigm for extrapolation, whether implicit or explicit. I prefer making that paradigm explicit.

  If you want to plug in raw experimental data, you are going to need data from people who are exactly like the patient in every way. Then, you will be relying on a paradigm for extrapolation which claims that the conditional counterfactual risks (rather than the magnitude of the effect) can be extrapolated from the study to the patient. It is a different paradigm, and one that can only be justified if the conditioning set includes every cause of the outcome.

  In my view, this is completely unrealistic. I prefer a paradigm for extrapolation that aims to extrapolate the scale-specific magnitude of the effect. If this is the goal, our conditioning set only needs to include those covariates that predict the magnitude of the effect of treatment, which is a small subset of all covariates that cause the outcome.

  On this specific point, my view is consistent with almost all thinking in medical statistics, with the exception of some very recent work in causal modeling (who prefer the approach based on counterfactual risks). My disagreement with this work in causal modeling is at the core of my last discussion about this on Less Wrong. See for example “Effect Heterogeneity and External Validity in Medicine” and the European Journal of Epidemiology paper that it links to

• Anders_HJun 14, 2021, 6:45 PM

  6 points

  in reply to: JenniferRM’s comment on: Shall we count the liv­ing or the dead?

  Suppose you summarize the effect of a drug using a relative risk (a multiplicative effect parameter relating the probability of the event if treated with the probability of the event if untreated), and consider this multiplicative parameter to represent the “magnitude of the effect”

  The natural thing for a clinician to do will be to assume that the magnitude of the effect is the same in their own patients. They will therefore rely on this specific scale for extrapolation from the study to their patients. However, those patients may have a different risk profile.

  When clinicians do this, they will make different predictions depending on whether the relative risk is based on the probability of the event, or the probability of the complement of the event.

  Sheps’ solution to this problem is the same as mine: If the intervention results in a decrease to the risk of the outcome, you should use the probability of the event to construct the relative risk, whereas if the intervention increases the risk of the event, you should use the probability of the complement of the event

• Anders_HJun 14, 2021, 1:14 PM

  12 points

  in reply to: George3d6’s comment on: Shall we count the liv­ing or the dead?

  No. This is not about interpretation of probabilities. It is about choosing what aspect of reality to rely on for extrapolation. You will get different extrapolations depending on whether you rely on a risk ratio, a risk difference or an odds ratio. This will lead to real differences in predictions for what happens under intervention.

  Even if clinical decisions are entirely left to an algorithm, the algorithm will need to select a mathematical object to rely on for extrapolation. The person who writes the algorithm needs to tell the algorithm what to use, and the answer to that question is contested. This paper contributes to that discussion, and proposes a concrete solution. One that has been known for 65 years, but never used in practice.

Shall we count the liv­ing or the dead?

• Anders_HOct 29, 2019, 9:17 AM

  8 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  Some time in the next week I’ll write up a post with a few full examples (including the one from Robins, Hernan and Wasserman), and explain in a bit more detail.

  I look forward to reading it. To be honest: Knowing these authors, I’d be surprised if you have found an error that breaks their argument.

  We are now discussing questions that are so far outside of my expertise that I do not have the ability to independently evaluate the arguments, so I am unlikely to contribute further to this particular subthread (i.e. to the discussion about whether there exists an obvious and superior Bayesian solution to the problem I am trying to solve).

• Anders_HOct 27, 2019, 1:15 PM

  6 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  I don’t have a great reference for this.

  A place to start might be Judea Pearl’s essay “Why I’m only half-Bayesian” at https://​​ftp.cs.ucla.edu/​​pub/​​stat_ser/​​r284-reprint.pdf . If you look at his Twitter account at @yudapearl, you will also see numerous tweets where he refers to Bayes Theorem as a “trivial identity” and where he talks about Bayesian statistics as “spraying priors on everything”. See for example https://​​twitter.com/​​yudapearl/​​status/​​1143118757126000640 and his discussions with Frank Harrell.

  Another good read may be Robins, Hernan and Wasserman’s letter to the editor at Biometrics, https://​​www.ncbi.nlm.nih.gov/​​pmc/​​articles/​​PMC4667748/​​ . While that letter is not about graphical models, the propensity scores/​​marginal structural models are mathematically very closely related. The main argument in that letter (which was originally a blog post) has been discussed on Less Wrong before; I am trying to find the discussion, it may be this link https://​​www.lesswrong.com/​​posts/​​xdh5FPMYYGGX7PBKj/​​the-trouble-with-bayes-draft

  From my perspective, as someone who is not well trained in Bayesian methods and does not pretend to understand the issue well, I just observe that methodological work on causal models very rarely uses Bayesian statistics, that I myself do not see an obvious way to integrate it, and that most of the smart people working on causal inference appear to be skeptical of such attempts

• Anders_HOct 27, 2019, 9:37 AM

  6 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  Ok, I think that’s the main issue here. As a criticism of Pearl and Bareinboim, I agree this is basically valid. That said, I’d still say that throwing out DAGs is a terrible way to handle the issue—Bayesian inference with DAGs is the right approach for this sort of problem.

  I am not throwing out DAGs. I am just claiming that the particular aspect of reality that I think justifies extrapolation cannot be represented on a standard DAG. While I formalized my causal model for these aspects of reality without using graphs, I am confident that there exists a way to represent the same structural constraints in a DAG model. It is just that nobody has done it yet.

  As for combining Bayesian inference and DAGs: This is one of those ideas that sounds great in principle, but where the details get very messy. I don’t have a good enough understanding of Bayesian statistics to make the argument in full, but I do know that very smart people have tried to combine it with causal models and concluded that it doesn’t work. Bayesianism therefore plays essentially no role in the causal modelling literature. If you believe you have an obvious solution to this, I recommend you write it up and submit to a journal, because you will get a very impactful publication out of it.

  The equality of this parameter is not sufficient to make the prediction we want to make—the counterfactual is still underspecified. The survival ratio calculation will only be correct if a particular DAG and counterfactual apply, and will be incorrect otherwise.

  In a country where nobody plays Russian roulette, you have valid data on the distribution of outcomes under the scenario where nobody plays Russian roulette (due to simple consistency). In combination with knowledge about the survival ratio, this is sufficient to make a prediction for the distribution of outcomes in a counterfactual where everybody plays Russian roulette.

• Anders_HOct 26, 2019, 7:55 PM

  11 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  Identifiability, sure. But latents still aren’t a problem for either extrapolation or model testing, as long as we’re using Bayesian inference. We don’t need identifiability.

  I am not using Bayesian inference, and neither are Pearl and Bareinboim. Their graphical framework (“selection diagrams”) is very explicitly set up as model for reasoning about whether the causal effect in the target population is identified in terms of observed data from the study population and observed data from the target population. Such identification may succeed or fail depending on latent variables and depending on the causal structure of the selection diagram.

  I am confident that Pearl and Bareinboim would not disagree with me about the preceding paragraph. The point of disagreement is whether there are realistic ways to substantially reduce the set of variables that must be measured, by using background knowledge about the causal structure that cannot be represented on selection diagrams.

  The obvious causal model for the Russian roulette example is one with four nodes:

  first node indicating whether roulette is played

  second node, child of first, indicating whether roulette killed

  third node, child of second, indicating whether some other cause killed (can only happen if the person survived roulette)

  fourth node, death, child of second and third node

  This makes sense physically, has a well-defined counterfactual for Norway, and produces the risk difference calculation from the post. What information is missing?

  In my model of reality (and I am sure, in most other people’s model of reality), the third node has a wide range of unobserved latent ancestors. If the goal is to make inferences about the effect of Russian roulette in Russia using data from Russia, your analytic objective will be to find a set of nodes that d-separate the first node from the fourth node. You do not need to condition on the latent causes of the third node to achieve this (because those latent variables are not also causes of the first node- they cannot be, because the first node was randomized). The identification formula for the effect in Russia is therefore invariant to whether the latent causes of the third node are represented on the graph or not, and you therefore do not have to show them. The DAG model then represents a huge equivalence class of causal models; you can be agnostic between causal models within this equivalence class because the inferences are invariant between them.

  But if the goal is to make predictions about the effect in Norway using data from Russia, these latent variables suddenly become relevant. The goal is no longer to d-separate the fourth node from the first node, but to d-separate the fourth node from an indicator for whether a person lives in Russia or Norway. In the true data generating mechanism (i.e. in the reality that the model is trying to represent), there almost certainly are a substantial number of open paths between the indicator for whether a person lives in Norway or Russia and their risk of death. The only possible identification formula for the effect in Russia includes terms for distributions that are conditional on the latent variables. The effect in Norway is therefore not identified from the Russian data.

  The underlying structure of reality is still a DAG, it’s only our information about reality which will be non-DAG-shaped. DAGs show the causal structure

  I agree that reality is generated by a structure that looks something like a directed acyclic graph. But that does not mean that all significant aspects of reality can be modeled using Pearl’s specific operationalization of causal DAGs/​selection diagrams.

  Any attempt to extrapolate from Russia to Norway is going to depend on a background belief that some aspect of the data generating structure is equal between the countries. In the case of Russian roulette, I argue that the natural choice of mathematical object to hang our claims to structural equality on, is the parameter that takes the value ⁵⁄₆ in both countries.

  In DAG terms, you can think of the data generating mechanism for node 4 as responding to a property of the path 1->2->4. In particular, this path forces the quantities Pr(Fourth node =0 | do(First node=1)) and Pr(Fourth node =0 | do(First node=0)) to be related by a factor of ⁵⁄₆ in both countries. Reality still has a DAG structure, but you won’t find a way to encode the figure ⁵⁄₆ in a causal model based only on selection diagrams. Without a way to encode a parameter that takes the value ⁵⁄₆, you have to take a long detour where you collect a truckload of data and measure all the latent variables.

• Anders_HOct 26, 2019, 1:13 PM

  15 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  The key issue is that we’re asking a counterfactual question. The question itself will be underdefined without the context of a causal model. The Russian roulette hypothetical is a good example: “Our goal is to find out what happens in Norway if everyone took up playing Russian roulette once a year”. What does this actually mean? Are we asking what would happen if some mad dictator forced everyone to play Russian roulette? Or if some Russian roulette social media craze caught on? Or if people became suicidal en-masse and Russian roulette became popular accordingly? These are different counterfactuals, and the answer will be different depending on which of these we’re talking about. We need the machinery of counterfactuals—and therefore the machinery of causal models—in order to define what we mean at all by “what happens in Norway if everyone took up playing Russian roulette once a year”. That counterfactual only makes sense at all in the context of a causal model, and is underdefined otherwise.

  I absolutely agree that this is a counterfactual question. I am using the machinery of counterfactuals and causal models, just a different causal model from the one you and Pearl prefer. In this case, I had in mind a situation that is roughly equivalent to a mad dictator forcing everyone to play Russian roulette, but the underspecified details are not all that important to the argument I am making.

  I assume by “unmeasured causes” you mean latent variables - i.e. variables in the causal graph which happened to not be observed. A causal diagram framework can handle latent variables just fine; there is no fundamental reason why every variable needs to be measured. Latent variables are a pain computationally, but they pose no fundamental problem mathematically.

  This is straight up wrong, and on this particular point the causal inference establishment is on my side, not yours. For example, if there are backdoor paths that cannot be closed without conditioning on a latent variable, then the causal effect is not identified and there is no amount of computation that can get around this.

  Indeed, much of machine learning consists of causal models with latent variables.

  Much of machine learning gets causality wrong.

  Whether the treatment has an effect does not seem relevant here at all.

  It is relevant because it allows me to construct a very simple scenario where we have very strong intuition that extrapolation should work; yet Pearl’s selection diagram fails to make a prediction for the target population.

  No. My intuition very strongly says that 100% of the relevant structural information/​model can be directly captured by causal models, and that you’re just not used to encoding these sorts of intuitions into causal models. Indeed, counterfactuals are needed even to define what we mean, as in the Russian roulette example. The individual counterfactual distributions really are the thing we care about, and everything else is relevant only insofar as it approximates those counterfactual distributions in some situations.

  I agree that you can encode all structural information in causal models. I do not agree that all structural information can be encoded in DAGs, which are one particular type of causal model. There are several examples of background information about the causal structure, which are essential for identifiability and which cannot be encoded on standard DAGs. For example, monotonicity is necessary for instrumental variable identification.

  I am arguing that there is a special type of background information that is crucial for generalizability, and which cannot be encoded in Pearl/​Bareinboim’s causal diagrams for transportability. I therefore proposed a non-DAG causal model which is able to use this background structural knowledge. The Russian roulette example is an attempt to illustrate the nature of this class of background knowledge.

  This does not mean that it is impossible to make an extension of the causal DAG framework to encode the same information. I am just arguing that this is not what the Pearl/​Bareinboim selection diagram framework does.

  Overall, my impression is that you don’t actually understand how to build causal models, and you are very confused about their applicability and limitations.

  I did specifically invoke Crocker’s Rules, so I’d like to thank you for this feedback.

  Of course, I think you are wrong about this. I dislike appeals to authority, but I would like to point out that I have a doctoral degree in epidemiologic methodology from Harvard, and that my thesis advisors were genuine thought leaders in causal modelling. I also want to point out that both my papers on this topic have been reviewed by editors and peer-reviewers with a deep understanding of causal models.

  This does of course not necessarily mean that you are wrong. It does however mean that I think you should adjust your priors and truly try to understand my argument before you reach such a strong posterior.

  If you genuinely have found a flaw in my argument, I’d like you to state it explicitly rather than just claim that I don’t understand causal models. In a hypothetical world in which I am wrong, I would very much like to know about it, as it would allow me to move on and work on something else.

• Anders_HOct 26, 2019, 8:41 AM

  8 points

  in reply to: johnswentworth’s comment on: Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

  I am curious why you think the approach based on causal diagrams is obviously correct. Would you be able to unpack this for me?

  Does it not bother you that this approach fails to find a solution (i.e. won’t make any predictions at all) if there are unmeasured causes of the outcome, even if treatment has no effect?

  Does it not bother you that it fails to find a solution to the Russian roulette example, because the approach insists on treating “what happens if treated” and “what happens if untreated” as separate problems, and therefore fails to make use of information about how much the outcomes differs by between the two treatment options?

  Does it not seem useful to have an alternative approach that is able make use of all the intuition that says we should be able to make such extrapolations? An alternative approach that formalizes the intuition that led all the pre-Pearl literature to consider the problem in terms of the magnitude of the effect, not in terms of the individual counterfactual distributions?

Effect het­ero­gene­ity and ex­ter­nal val­idity in medicine

Coun­ter­fac­tual out­come state tran­si­tion parameters

• Anders_HDec 2, 2017, 6:56 PM

  3 points

  in reply to: DanB’s comment on: The New Rid­dle of In­duc­tion: Neu­tral and Rel­a­tive Per­spec­tives on Color

  In my view, “the problem of induction” is just a bunch of philosophers obsessing over the fact that induction is not deduction, and that you therefore cannot predict the future with logical certainty. This is true, but not very interesting. We should instead spend our energy thinking about how to make better predictions, and how we can evaluate how much confidence to have in our predictions. I agree with you that the fields you mention have made immense progress on that.

  I am not convinced that computer programs are immune to Goodmans point. AI agents have ontologies, and their predictions will depend on that ontology. Two agents with different ontologies but the same data can reach different conclusions, and unless they have access to their source code, it is not obvious that they will be able to figure out which one is right.

  Consider two humans who are both writing computer functions. Both the “green” and the “grue” programmer will believe that their perspective is the neutral one, and therefore write a simple program that takes light wavelength as input and outputs a constant color predicate. The difference is that one of them will be surprised after time t, when suddenly the computer starts outputting different colors from their programmers experienced qualia. At that stage, we know which one of the programmers was wrong, but the point is that it might not be possible to predict this in advance.