The Unfriendly Superintelligence next door
Markets are powerful decentralized optimization engines—it is known. Liberals see the free market as a kind of optimizer run amuck, a dangerous superintelligence with simple non-human values that must be checked and constrained by the government—the friendly SI. Conservatives just reverse the narrative roles.
In some domains, where the incentive structure aligns with human values, the market works well. In our current framework, the market works best for producing gadgets. It does not work so well for pricing intangible information, and most specifically it is broken when it comes to health.
We treat health as just another gadget problem: something to be solved by pills. Health is really a problem of knowledge; it is a computational prediction problem. Drugs are useful only to the extent that you can package the results of new knowledge into a pill and patent it. If you can’t patent it, you can’t profit from it.
So the market is constrained to solve human health by coming up with new patentable designs for mass-producible physical objects which go into human bodies. Why did we add that constraint—thou should solve health, but thou shalt only use pills? (Ok technically the solutions don’t have to be ingestible, but that’s a detail.)
The gadget model works for gadgets because we know how gadgets work—we built them, after all. The central problem with health is that we do not completely understand how the human body works—we did not build it. Thus we should be using the market to figure out how the body works—completely—and arguably we should be allocating trillions of dollars towards that problem.
The market optimizer analogy runs deeper when we consider the complexity of instilling values into a market. Lawmakers cannot program the market with goals directly, so instead they attempt to engineer desireable behavior by ever more layers and layers of constraints. Lawmakers are deontologists.
As an example, consider the regulations on drug advertising. Big pharma is unsafe—its profit function does not encode anything like “maximize human health and happiness” (which of course itself is an oversimplification). If allowed to its own devices, there are strong incentives to sell subtly addictive drugs, to create elaborate hyped false advertising campaigns, etc. Thus all the deontological injunctions. I take that as a strong indicator of a poor solution—a value alignment failure.
What would healthcare look like in a world where we solved the alignment problem?
To solve the alignment problem, the market’s profit function must encode long term human health and happiness. This really is a mechanism design problem—its not something lawmakers are even remotely trained or qualified for. A full solution is naturally beyond the scope of a little blog post, but I will sketch out the general idea.
To encode health into a market utility function, first we create financial contracts with an expected value which captures long-term health. We can accomplish this with a long-term contract that generates positive cash flow when a human is healthy, and negative when unhealthy—basically an insurance contract. There is naturally much complexity in getting those contracts right, so that they measure what we really want. But assuming that is accomplished, the next step is pretty simple—we allow those contracts to trade freely on an open market.
There are some interesting failure modes and considerations that are mostly beyond scope but worth briefly mentioning. This system probably needs to be asymmetric. The transfers on poor health outcomes should partially go to cover medical payments, but it may be best to have a portion of the wealth simply go to nobody/everybody—just destroyed.
In this new framework, designing and patenting new drugs can still be profitable, but it is now put on even footing with preventive medicine. More importantly, the market can now actually allocate the correct resources towards long term research.
To make all this concrete, let’s use an example of a trillion dollar health question—one that our current system is especially ill-posed to solve:
What are the long-term health effects of abnormally low levels of solar radiation? What levels of sun exposure are ideal for human health?
This is a big important question, and you’ve probably read some of the hoopla and debate about vitamin D. I’m going to soon briefly summarize a general abstract theory, one that I would bet heavily on if we lived in a more rational world where such bets were possible.
In a sane world where health is solved by a proper computational market, I could make enormous—ridiculous really—amounts of money if I happened to be an early researcher who discovered the full health effects of sunlight. I would bet on my theory simply by buying up contracts for individuals/demographics who had the most health to gain by correcting their sunlight deficiency. I would then publicize the theory and evidence, and perhaps even raise a heap pile of money to create a strong marketing engine to help ensure that my investments—my patients—were taking the necessary actions to correct their sunlight deficiency. Naturally I would use complex machine learning models to guide the trading strategy.
Now, just as an example, here is the brief ‘pitch’ for sunlight.
If we go back and look across all of time, there is a mountain of evidence which more or less screams—proper sunlight is important to health. Heliotherapy has a long history.
Humans, like most mammals, and most other earth organisms in general, evolved under the sun. A priori we should expect that organisms will have some ‘genetic programs’ which take approximate measures of incident sunlight as an input. The serotonin → melatonin mediated blue-light pathway is an example of one such light detecting circuit which is useful for regulating the 24 hour circadian rhythm.
The vitamin D pathway has existed since the time of algae such as the Coccolithophore. It is a multi-stage pathway that can measure solar radiation over a range of temporal frequencies. It starts with synthesis of fat soluble cholecalciferiol which has a very long half life measured in months. [1] [2]
Cholecalciferiol (HL ~ months) becomes
25(OH)D (HL ~ 15 days) which finally becomes
1,25(OH)2 D (HL ~ 15 hours)
The main recognized role for this pathway in regards to human health—at least according to the current Wikipedia entry—is to enhance “the internal absorption of calcium, iron, magnesium, phosphate, and zinc”. Ponder that for a moment.
Interestingly, this pathway still works as a general solar clock and radiation detector for carnivores—as they can simply eat the precomputed measurement in their diet.
So, what is a long term sunlight detector useful for? One potential application could be deciding appropriate resource allocation towards DNA repair. Every time an organism is in the sun it is accumulating potentially catastrophic DNA damage that must be repaired when the cell next divides. We should expect that genetic programs would allocate resources to DNA repair and various related activities dependent upon estimates of solar radiation.
I should point out—just in case it isn’t obvious—that this general idea does not imply that cranking up the sunlight hormone to insane levels will lead to much better DNA/cellular repair. There are always tradeoffs, etc.
One other obvious use of a long term sunlight detector is to regulate general strategic metabolic decisions that depend on the seasonal clock—especially for organisms living far from the equator. During the summer when food is plentiful, the body can expect easy calories. As winter approaches calories become scarce and frugal strategies are expected.
So first off we’d expect to see a huge range of complex effects showing up as correlations between low vit D levels and various illnesses, and specifically illnesses connected to DNA damage (such as cancer) and or BMI.
Now it turns out that BMI itself is also strongly correlated with a huge range of health issues. So the first key question to focus on is the relationship between vit D and BMI. And—perhaps not surprisingly—there is pretty good evidence for such a correlation [3][4] , and this has been known for a while.
Now we get into the real debate. Numerous vit D supplement intervention studies have now been run, and the results are controversial. In general the vit D experts (such as my father, who started the vit D council, and publishes some related research[5]) say that the only studies that matter are those that supplement at high doses sufficient to elevate vit D levels into a ‘proper’ range which substitutes for sunlight, which in general requires 5000 IU day on average—depending completely on genetics and lifestyle (to the point that any one-size-fits all recommendation is probably terrible).
The mainstream basically ignores all that and funds studies at tiny RDA doses—say 400 IU or less—and then they do meta-analysis over those studies and conclude that their big meta-analysis, unsurprisingly, doesn’t show a statistically significant effect. However, these studies still show small effects. Often the meta-analysis is corrected for BMI, which of course also tends to remove any vit D effect, to the extent that low vit D/sunlight is a cause of both weight gain and a bunch of other stuff.
So let’s look at two studies for vit D and weight loss.
First, this recent 2015 study of 400 overweight Italians (sorry the actual paper doesn’t appear to be available yet) tested vit D supplementation for weight loss. The 3 groups were (0 IU/day, ~1,000 IU / day, ~3,000 IU/day). The observed average weight loss was (1 kg, 3.8 kg, 5.4 kg). I don’t know if the 0 IU group received a placebo. Regardless, it looks promising.
On the other hand, this 2013 meta-analysis of 9 studies with 1651 adults total (mainly women) supposedly found no significant weight loss effect for vit D. However, the studies used between 200 IU/day to 1,100 IU/day, with most between 200 to 400 IU. Five studies used calcium, five also showed weight loss (not necessarily the same—unclear). This does not show—at all—what the study claims in its abstract.
In general, medical researchers should not be doing statistics. That is a job for the tech industry.
Now the vit D and sunlight issue is complex, and it will take much research to really work out all of what is going on. The current medical system does not appear to be handling this well—why? Because there is insufficient financial motivation.
Is Big Pharma interested in the sunlight/vit D question? Well yes—but only to the extent that they can create a patentable analogue! The various vit D analogue drugs developed or in development is evidence that Big Pharma is at least paying attention. But assuming that the sunlight hypothesis is mainly correct, there is very little profit in actually fixing the real problem.
There is probably more to sunlight that just vit D and serotonin/melatonin. Consider the interesting correlation between birth month and a number of disease conditions[6]. Perhaps there is a little grain of truth to astrology after all.
Thus concludes my little vit D pitch.
In a more sane world I would have already bet on the general theory. In a really sane world it would have been solved well before I would expect to make any profitable trade. In that rational world you could actually trust health advertising, because you’d know that health advertisers are strongly financially motivated to convince you of things actually truly important for your health.
Instead of charging by the hour or per treatment, like a mechanic, doctors and healthcare companies should literally invest in their patients long-term health, and profit from improvements to long term outcomes. The sunlight health connection is a trillion dollar question in terms of medical value, but not in terms of exploitable profits in today’s reality. In a properly constructed market, there would be enormous resources allocated to answer these questions, flowing into legions of profit motivated startups that could generate billions trading on computational health financial markets, all without selling any gadgets.
So in conclusion: the market could solve health, but only if we allowed it to and only if we setup appropriate financial mechanisms to encode the correct value function. This is the UFAI problem next door.
Robin Hanson proposed much the same over 20 years ago in “Buy Health, Not Health Care”.
Interesting read. In the same vein. What I was imagining is a computational market, relying on the ability to do lots of complex trades at high speeds, and AI/ML. But much of that difference is explained by the 20 years.
I also reviewed some of his prototype code for a combinatorial prediction market around 10 years ago. I agree that these are promising ideas and I liked this post a lot.
As someone living in a universal/governmental healthcare country, I think we are doing this. If I am healthy and working, I am an asset for the state, pay taxes / social security. If I am ill or disabled, they gotta pay me. If I am dead, I don’t pay them.
Of course it is not ideal first of all because of the usual problem of government: politicians, bureaucrats don’t get a dividend from the profits of the state, they are not incentivized to maximize profitability. Secondarily, there are some incentive pitfalls like I am cheaper for them dead than pulling disability pay. Once it would look likely I will never work again their incentive would be providing zero healthcare.
So while it is not ideal, the basic idea of people paying something to an organization every month or year when they are healthy and working, and the healthcare costs are paid by that organization so they want to keep their client healthy and working and paying is there, and it can be tweaked. Part of the story is that people even when retired should keep paying. From this angle life insurance is better than social security.
However I think there is no workaround for the fact that once you have 5 years to live and extending that to 10 years costs a lot, whatever tax or insurance premiums you would pay in the second 5 years would not cover it and thus the organization has no incentive to extend your life. This can only be done by strict contracts or by politics. A third option is kids.
Let’s go a big sci-fi here, we make a pill that extends female fertility up to about 60 easy, and thus we can assume most people will have kids again because it can just as treated like an early retirement.
The point is, if people have kids, you can treat families as immortal or long-lived persons. You can work out a scheme that if dad’s life is not extended the kids will take their insurance elsewhere.
There’s research suggesting that in developing countries, increased healthcare spending doesn’t improve health outcomes like longevity. Don’t know how good the research is though.
I wonder what Yvain/Scott would have to say about the vitamin D pitch.
I had read that article and wanted to explicitly mention it but couldn’t re-find it in time, he has so many blogs post that mention vit D.
Yvain’s view seems similar to the viewpoint I critiqued in the article. He doesn’t care what dose the study is using, ethnicity/genetics, or if it is likely to raise OHD levels sufficiently at all.
The meta-analysis studies he links to are of the kind I critiqued—it’s basically gerrymandering. These issues should be decided by largescale public ML competitions, not dark age statistics.
Did you read the New England paper? They specifically measured serum levels, did a very thorough sub group analysis (including ethnicity and many other possible factors), broke down the supplementation by dosage. And all within a 36,000 person double blinded randomized control trial.
I don’t think I need to point out the danger in creating a just-so story in favor of your belief then approvingly citing sources that agree with your conclusion, while slamming sources that don’t give you the results you want. Granted, the problem of certain medications only working on some groups is somewhat difficult and this is a very common complaint when studies show a null effect. Perhaps a medication really does only work on elderly Hispanic women—and its not just an instance of p-value hacking. However, when a massive amount of data shows that there is no discernible effect of supplementation across many groups, you have to decide when to set aside a hypothesis until there is a good reason to consider it again.
(of course, this is just the colorectal cancer case, but I think it illustrates that the medical research community is well aware of the issues you bring up. Epidemiology is hard and no one has ever claimed otherwise).
Could you cite the paper you mean?
Sorry here it is:
http://www.fp.ucalgary.ca/FMResidentSecure/Articles/Ca%20vit%20d%20and%20ca%20colon%20nejm%20feb%202006.pdf
Not sure. Is it the colon cancer one? I read the breast cancer one linked on his post. I dont believe/expect that vit D is going to make an impact for every illness or type of cancer. I expect the relationship will be complex, and will depend heavily on dietary factors and genetics (for example there is strong evidence for various climdate adaptations—far northern peoples need much less vit D, etc.) I picked the breast cancer paper because I have a vague memory of hearing a very convincing talk from a breast cancer doctor about the evidence for the bio mechanisms between the low vit D and breast cancer.
From the summary of the breast cancer paper yvain linked to:
Methods Postmenopausal women (N = 36 282) who were enrolled in a Women’s Health Initiative clinical trial were randomly assigned to 1000 mg of elemental calcium with 400 IU of vitamin D3 daily or placebo for a mean of 7.0 years
Results Invasive breast cancer incidence was similar in the two groups (528 supplement vs 546 placebo; hazard ratio = 0.96; 95% confidence interval = 0.85 to 1.09). In the nested case–control study, no effect of supplement group assignment on breast cancer risk was seen. Baseline 25-hydroxyvitamin D levels were modestly correlated with total vitamin D intake (diet and supplements) (r = 0.19, P < .001) and were higher among women with lower BMI and higher recreational physical activity (both P < .001). Baseline 25-hydroxyvitamin D levels were not associated with breast cancer risk in analyses that were adjusted for BMI and physical activity (Ptrend = .20).
So they used only 400 IU, and even then the breast cancer incidence was 528 vs 546 for 400 IU vs 0 IU. There was no effect only when they “adjusted for BMI”—of course—as some other studies have suggested, vit D has a potential weight loss effect. The 400 IU didn’t make a huge diff on OHD levels.
This study is supportive of vit D potentially having a significant effect on breast cancer, and it is reasonable evidence against the null hypothesis (no effect). Instead here is their conclusion:
Conclusions Calcium and vitamin D supplementation did not reduce invasive breast cancer incidence in postmenopausal women. In addition, 25-hydroxyvitamin D levels were not associated with subsequent breast cancer risk. These findings do not support a relationship between total vitamin D intake and 25-hydroxyvitamin D levels with breast cancer risk.
That is not justifiable, nor is it even an accurate summary of their own data.
Yes I meant the colon cancer paper (I should have stated at the beginning rather than the end), but the breast cancer paper was from the same experiment, and as such also addresses you initial concerns about effects on different subgroups and measuring serum levels.
The difference in incidence was not statistically significant. Random chance could easily result in this level of variation. Regarding the dosage, yes you can only make conclusions about what you measure. This trial was started in the 90′s, and since then there have been some recommendations for higher doses. However, the authors are not unaware of this and point out that many women in the trial were supplementing with additional Vitamin D, and after analyzing that, there was still no detectable effect:
Perhaps a still higher dose is need to see any effect. But this would require positing a very non-linear model. Is it possible? Sure, but this complaint can always be made.
Regarding BMI, the authors are also aware of this argument—see the quote below. I’d like to point out that the BMI numbers were baseline which is not evidence that vitamin D supplementation causes weight loss. It is consistent with the hypothesis that healthier lifestyles result in both higher vitamin D levels (due to sun exposure with exercise) and lower BMI.
You can argue that the question isn’t yet settled. I think that is defensible—maybe we really do need much higher doses. But you cannot argue that the researchers haven’t done their homework or that they aren’t aware of what their data is capable or incapable of showing.
This is primitive frequentist statistics.
If we could repeat this same experiment with a very similar distribution of patients and other factors (or if the trial was still ongoing), and it came down to a bet on which group would have higher cancer deaths—would you accept an even money bet that the placebo group would have less cancer deaths? ie—would you bet against me?
Whether 3% more deaths in the placebo group is significant or not for a sample size of 1000, and an average supplement level which is only 10% of what the vit D experts believe is required to replace sunlight depends completely on your priors and your statistical model. There is nothing innate or inherently correct about the standard puny little models used to determine ‘statistical significance’.
Thinking in terms of prediction markets helps clarify these issues—the best model is the one that ends up sitting on the big heap pile of money.
Sure—or maybe not. It is still evidence that favors a vit D effect for breast cancer.
Half of the women reported taking an additional 400 IU, which—even with full compliance—only raises the average up to 600 IU.
The main key point is that the range of measured OHD levels was between 30 to 67 ish and didn’t have a strong correlation to the supplement—they failed to raise the OHD levels. That is key. All of the studies which show a health effect also significantly shift the OHD levels. If your supplement isn’t shifting OHD levels much, why would think it would do much of anything?
The general theory predicts health effects in 1.) certain genetic subpopulations, who 2.) have unnaturally low levels (say <30 for caucasians), and 3. then raise those levels up to evolutionary ideal for their genetics (perhaps 50-60 for caucasians). The exact numbers are just hyperparameters—the point of the studies should be to refine them into a predictive model.
The completely honest conclusion for their abstract should have been something like this:
Conclusions Calcium and low doses of vitamin D supplementation did not significantly reduce (> 3%) invasive breast cancer incidence in postmenopausal women. In addition, 25-hydroxyvitamin D levels were not associated with subsequent breast cancer risk after we adjusted for body mass index. It is difficult to make definitive conclusions from these findings.
Compare that to the conclusion they actually wrote. Then the telephone game begins, and people begin quoting this study as one which somehow ‘proves’ vit D supplementation has no effect on breast cancer . …
If you want to bet there the VITAL study going on: http://www.vitalstudy.org/VitalSigns.html 25,000 subjects 2000 UI Vitamin D3 and upcoming mortality data.
I also created a prediction book entry a while ago: http://predictionbook.com/predictions/14426
I’m sorry, I’m really not interested in getting into an ideological debate or whether frequentist or bayesian statistics is “better”—if you think that frequentist methods are worthless, then the inferential gap is too wide to begin to bridge.
Agreed—I do believe that Frequentist methods are primitive compared to modern machine learning.
Also, I don’t even have a strong opinion on whether a few years of vit D supplementation in the elderly is going to make a big difference in many health outcomes like cancer risk—the correct comparison is between a lifetime of adequate vit D levels vs a lifetime of inadequacy. I don’t suspect that correcting it late in life is going to avoid most of the cancer risk.
There is literally a mountain of evidence for health effects of sunlight. Reality doesn’t reduce to some trivial little logical statement that you can test with a meta-analysis.
I’m sure you can see the problem with the argument that because sunlight increases vitamin D and sunlight has some beneficial health effects, then vitamin D supplementation has the same health benefits. This is a reasonable hypothesis to test, but it cannot be asserted as is without checking each step in the casual pathway. Without this crucial step it is nothing more than an just so story.
Vitamin D is the main known mechanism by which sunlight can effect health. That does not mean it is the only mechanism—there is some recent evidence for nitric oxide effects, for example—it just means it is the mechanism that is most understood and the most potent from what we know now.
This is not a just so story—research from the last 5-8 years or so has shown how vit D regulates gene expression in a host of tissues. The general theory that it is an input into a very large number of gene programs/networks is extremely solid.
Based on that foundation, the prior that vit D supplementation would have zero health effects should be very small. Of course that does not imply that the health effects are always positive!
Instead it merely implies that if you have a study which shows no effects—than by overwhelming probability—either they used too small a dose, or they happened to test something specific that vit D does not effect, or they made some more fundamental error.
“In some domains, where the incentive structure aligns with human values, the market works well. In our current framework, the market works best for producing gadgets. It does not work so well for pricing intangible information, and most specifically it is broken when it comes to health.”
Health care, as an example of a problem “market”?
In the US, health care is largely just a shake down by various licensed monopolies engaged in government enabled and enforced rent seeking.
We’d be free.
In a US that had been free for the last 20 years, google would be analyzing the data from my weekly, if not daily microfluidic samples, from real time continuous information provided by sensors throughout the day, from my dna, and doing the same for millions of other people. We’d share our data so that google could make better recommendations for achieving our health goals, and with the mass of data everyone shares, google would be able to make those predictions.
The only market “alignment” required is allowing a market in health care to actually exist.
Yes, and? A market exists where a market exists. Any moral or ideological properties that might make a market “pure” or “impure” are just attributed to it by ideologues.
You seem to be confusing Google with a Friendly singleton. Of course, once you get into Singletons, it’s easier to have a democratic government function as one, anyway.
The properties are what make it a market, a chimpanzee, or a turnip.
“The beginning of wisdom is to call things by their proper name.” ― Confucius
Google and government have different powers and incentives.
If the US were “free” you’d have a gazillion snake oil salesmen,
We do anyways.
And I’d be free to buy from them, or not.
And if they committed fraud in the sale, I would have legal recourse.
If you were still alive.
Historically, this didn’t work out well. You know, back when the snake oil salesmen were literal and selling real snake oil, cocaine, and various low-dose toxic extracts. (I believe similar things happen in China today, but it’s more slanted toward traditional medicine and thus less likely to be toxic.)
That’s broadly the structure of taxpayer supported public healthcare system. The government is incentivised to keep people alive and paying tax, and disincentivised to treat people unnecessarily. I can vouch that under that kind of system, the authorities aren’t shy about promoting preventative measures.
I generally agree that the government is more value aligned, but it is also much less efficient.
In particular it has trouble with fine grained credit assignment. It works well for running a military, it doesn’t work as well in the modern era for research—where anybody with some smarts, the internet, and luck can come up with an idea that is potentially worth billions—without any credentials.
Do you want rewards as a means of getting good outcomes, or as an end in itself?
yep, alive and paying tax or dead quickly if they’re going to die.
There’s a reason there was a big push for anti-smoking measures in recent years. Cancer treatments got better, people with lung cancer suddenly lived longer and cost more so the incentive to stop people smoking grew.
Better aligned but not perfectly.
Also it is largely a command economy setup.
Better, of course, is still better than worse. “Perfectly aligned”, I’d say, is an overly strong claim to make for anything in this mortal world, ahaha, that was not explicitly designed and verified to be perfectly aligned, or at least aligned in-the-limit of increasing resources.
There are countries with non-command economy public healthcare. I lived in one (Israel).
It’s also important to note that markets, in the sense of allocatively-efficient combinatorial price-seeking and price-taking as such, are not the magic super sauce of Western economies as we know them. As far as economists were able to study, the magic super sauce is actually just allowing firms to enter and exit business as they please, enabling economic agents to plan reliably, and containing liability when things go wrong.
Command economies violate that first guideline, which takes away the ability of an economy to explore a combinatorially-large solution space in parallel.
The last problem was once “solved” via limited liability corporations, but nowadays bailout/hostage capitalism and the increasing financialization of Western economies has created a previously-unrecognized creature: distinctly capitalist economies defined by catering to a small number of agents who can punish everyone else with toxic liabilities when they don’t get their upside.
The middle problem is where different societies have usually had to fudge things. A strong sense of private property often helps, but an overly strong institution of private property allows hostage-taking (as described above with banks, or as is the case with natural-resource rents, or as witnessed in any Bay Area urban planning meeting). Things like collective bargaining, universal health care, or Flexicurity seem to run against the grain of private property, but actually help increase the total planning reliability across society, and certainly people like having them. There doesn’t seem to be a secret sauce for building institutional structures that give people the Freedom To Optimize.
There’s a great chart of the popularity of tobacco, and of various modes of consuming it, on the first page of this paper:
http://cancercontrol.cancer.gov/brp/tcrb/monographs/8/m8_2.pdf
There’s also a timeline of related events on page 16.
Peak tobacco use corresponded with the period where scientific studies were establishing that smoking causes cancer and various other diseases. It was several years into the decline of smoking popularity before “nonsmoker’s rights” (i.e. measures against smoking in public) started taking off.
Unfortunately I don’t think that’s true:
source
I can see that private health insurance companies wouldn’t want to take in bad risks. They don’t in some US states, and are forced to by the federal government in others, I can also see that a profit driven GovCo would behave like a giant private insurer. But actual in-GovCo governments are incentivised to provide universally access to healthcare as they do to the law and education. I can vouch that where you have public healthcare, any hint that some group is excluded creates a stink. Health insurance provides better incentives than piecemeal provision, but public healthcare has better incentives than both.
un-GovCo, I believe?
I appreciate the point and agree with it, but in fairness to the financial prudence folks if the government were to start executing citizens that would have major costs down the line. I would not expect a government concerned with financial prudence to do such executions if they were at all intelligent. It’s cheaper to give unproductive people welfare than to round them all up and kill them, once you take matters of political economy into consideration on the accounting sheet.
I really like this as an example of incentive misalignment, and I think it should be in the community’s core materials on that subject.
That said, the ideas about healthcare contracts to fix the alignment problem could use some fleshing out. So here’s my attempt at fleshing them out.
General properties of the problem:
We want an open market in healthcare ideas. Anybody should be able to invest in whatever idea they have.
Ideas which do not work should cost the investor, ideas which do work should reward the investor.
The ideas must be tried on some actual people. Those people, unavoidably, bear some risk, and must be incentivized to take that risk.
Solution: First, the cost to incentivize people to try ideas must be paid by someone. The investors are a natural choice here. So imagine that the investors offer payment to people to try their ideas. To avoid selection issues, the investors may specify that people must fulfill certain criteria in order to be eligible. Anyone who meets the criteria can sign up for whatever the investors are pushing. This would be sort of like an open market for clinical trials, except that it can scale to any number of people and to long times.
For their investment, the investors would get a contract. Under that contract, they get some regular payments for as long as their participants see the benefits the investors claimed and don’t see serious problems. That payment ultimately needs to come from the participants; all participants pay to participate in the market as a whole. If the market is government-managed, then everyone is a participant and their payment comes from taxes.
As an example, consider a cure for prostate cancer. The investors put out their cure at some price, paying people to try their particular idea. Only those with prostate cancer are eligible. All participants get paid by the investors. As soon as a participant’s cancer clears, the investors get paid a big chunk of money.Whenever the participant has health issues later in life, the investors have to pay some money back, regardless of what those later issues are. Presumably, more serious issues would be more expensive, and the original contract might specify an extra payment if the prostate cancer returns. (There would be a secondary market for investors to insure against normal later-life health risks, but investors would still need to handle more-than-normal problems, and would get paid for less-than-normal problems).
Now an even more interesting example: a new diet. The problem with monetizing a new diet is that, once word gets out, anyone can just try it without having to pay. Thus the prevalence of pills: pills can be patented and monetized. But with this market structure, the investors are PAYING people to participate. Anyone who’s trying the diet will WANT to sign up, because they get paid. The investors can then reap the benefits (if any) over the person’s lifetime.
Finally, a preventative example. Once again, we have a prostate cancer treatment, but this time it is preventative rather than a cure. This time, prostate cancer is not required for participation. The investors pay the participants for their effort, and make money for every prostate-cancer-free year of every participant’s life. Again, there is more complexity around paying for any later problems, insuring those payments, etc… but that’s all standardized stuff and the associated costs are known in advance.
I like the idea of how to use markets to ‘solve’ health and esp. the illustrative example. Reminds me of David Friedman’s approach so solve law (see Anarchy and Efficient Law).
One nitpick regarding the post: It is really two: The health market and the vitamin D story. These are even clearly consecutive. I also do not see a clear connection between them except in your narrative. Why not post separatedly?
ADDED: And the title is misleading. It only relates to the opener, not the meat of the post.
David Friedman has been on the web forever—I read his stuff back in the 90′s I think and it influenced me.
Yes it is two posts in one. I wanted an example to more concretely illustrate something that is really an information problem that a computational market could work better for. This ended up being it’s own post inside the post as I got into writing it.
Got a better title idea? It relates to the overall metaphor.
While I’m glad to see someone writing up critiques of markets and capitalism in language LW will understand, really really incredibly glad in fact, I do have one nit to pick:
Not only is politics spiders, this is a simplistic strawman of both positions at once. The article would be better for cutting out the meta-level mention of politics and sticking to its object-level claims of fact.
But overall, you have all my upvotes and I wish to subscribe to your newsletter.
Noted!
Haha—very true. I realized it was a dramatic oversimplification, but I wanted to start by establishing the frame of analyzing a politicized issue from an .. apolitical stance. Thus I decided to equally critique both sides of the political spectrum—in a sentence or two. (or perhaps this is all post-hoc rationalization! And perhaps there is no functional difference.)
The health care system is not so unfriendly it will kill us all. Maybe it is time to stop using unfriendly to mean a wide variety of conditions.
It’s unfriendly in the sense that its values are not aligned with ours. I thought that’s what the U in UFAI meant. Luckily, it’s either not so powerful or not so misaligned that it will kill us all. But I think it’s more or less reasonable to say it’s unfriendly.
This comment is a follow-up to both this post, and Buy Health, Not Health Care.
Possible job title in a saner version of the future: Lifestyle Advisor.
“Paenacea Health, America’s leading Life Maintenance Organization seeks Lifestyle Advisors to work in our New York, Los Angeles, Phoenix, and Austin field offices. Lifestyle Advisors grow to develop relationships with 40-80 members, and will be responsible for contacting members on a weekly basis to encourage them to adopt better health practices. Upon joining our team, Lifestyle Advisors will undergo a three week training program, in which candidates will be familiarized with our approach to assessing the weighted importance of different behavioral changes to the life expectancy extension of our members.
All Paenacea Health, we truly value our members’ health—which means that all members are assigned a Lifestyle Advisor. Ideal candidates for this position will have excellent interpersonal skills, and will be able to successfully coach Paenacea members through each step of processes like quitting smoking or establishing an exercise routine. Candidates are expected to have the persistence and vision necessary to proactively and insistently initiate contact with members who have not adopted adequate health practices within a short period of joining Paenacea Health...”
Thanks for this post—very interesting.
One question:
Are you claiming that the tech industry is better at stats, or could be better at stats, if it were somehow decentralized? If the former, what’s your evidence?
Standard statistics is just primitive machine learning. Any question that you have about data is best answered by a well structured machine learning system/market, not old-school statistics. This is especially important for big complex health questions.
To make this more concrete:
First, a huge amount of health data is collected. Really all of the health data we have should be put into public repositories (anonymized/protected by the gov). The gov should then fund largescale prediction contests based on this data (kaggle is one implementation, prediction markets are more advanced form). For example, predicting how many people in X demographic cluster receiving Y interventions will receive a breast cancer diagnosis in the next year, or next month, etc.
The models that make the best predictions can then be used to predict farther into the future, to predict the outcome of new interventions, etc.
One problem with that idea is that it is really hard to effectively anonymize a large data set.
That article illustrates a general common failure mode in trying to prove something is impossible or really hard simply by listing a few examples of previous failed attempts to solve the problem.
Every attempt at manned flight failed until the first success, etc.
The examples they list make trivial errors. The health data obviously doesn’t need to contain any bits of someone’s SSN, phone number, or their exact address. Those are just obvious failures. Also, in general you can achieve arbitrary anonymization success by averaging samples—so you cluster individuals into groups of N based on similar demographics, and average all data across the sample cluster. This actually doesn’t necessarily reduce the data’s effectiveness that much, as it tends to average out noise. It’s related to batch methods in machine learning which average across a number of samples before doing any inference steps. All the big ANN systems are trained with batch averaging over hundreds or more of samples.
Those trivial errors weren’t made by them.
In general anonymisation is hard. Hard enough that there’s an interest of lawmakers to regulate it which makes it hard to regulate.
At the moment it seems like Google, Microsoft and Apple all build their health data storages, and then those companies put their own machine learning people on the problem.
This is true. And to follow with your example, just as some failed attempts at manned flight resulted in serious injury or death of the pilot, similarly some failed attempts to anonymize a large data set that is then released to the public will cause an unreversible loss of privacy to thousands or millions of people.
I never claimed that the article had proven effective anonymization impossible just as (I don’t think that) you are claiming that it is proven possible. My claim is that we need to balance the benefits of such data releases against the risk to the privacy of the people whose data are released.
You mentioned a potentially effective way to strengthen the anonymization:
In my experience, when you cluster and average data, you have to make some assumptions about the sorts of questions researchers will be trying to answer. Even if machine learning tends to average across batches, the decision about how to cluster the data is usually a function of the kinds of questions you are trying to answer with the data. It seems to me raw data is more useful than clustered, averaged data, because it has not presupposed the types of questions that will be asked. That said, averaging may be necessary if we are to release anonymized data safely, but it is a tradeoff. This tradeoff between safety and usefulness is one point that the Ars Technica article was making.
Yes, agreed with just about all of that.
Yes there is probably a fundamental information tradeoff between anonymization and data effectiveness, but it isn’t clear that this will be much of a limiter in practice.
Secondly, people should be able to opt-in to various levels of anonymization risk, and perhaps that could be tied to financial incentives, so that you can effectively sell your data to some degree.
As far as I understand the Microsoft-Band can already measure the amount of UV radiation a person perceives. If we are lucky that will become the standard for smart watches and we will get a lot of new data about sun exposure.
As far as Vitamin D research goes, a bunch of Quantified Self folks have found that for them taking vitamin D3 in the morning has good effects on sleep while taking it in the evening has bad effects on sleep.
The studies on Vitamin D don’t track timing at all. A bunch of Vitamin D studies likely fail to produce good effects because the Vitamin D is taken at the wrong time.
An app that knows your sleep data and your sun exposure could calculate the amount of Vitamin D that you should take and when you should take it..
Unfortunately it seems like sunlight causes wrinkles
That could be huge. It would be interesting if the gov could fund research by something more like kaggle, where they collect a mountain of data, and then they outsource the construction of models that best predict health outcomes. We need to move past simplistic statistics and meta-analysis.
I like this analogy. So basically, how do you want to balance the power between your two overlords, one much much smarter than you but with non-human values, and the other much dumber than you but with human (mostly) values.
Who says the state is dumb? It created the market, after all.
What does that mean? Across history, there is a whole spectrum of complexity of both markets and governmental arrangements, and against that backdrop it’s not clear that it means anything to talk about “the state” or “the market”, still less one creating the other. You might as well say that “the market” created “the state”, and I expect that a historian could argue either thesis with equal facility.
And some people would like to make it sit down and write “I will not conjure up what I can’t control” a thousand times for this. But I, for one, welcome our efficient market overlords!
TL;DR: Let life- and health-insurance companies, which are interested in minimizing insurance claims, not maximizing health-related sales, set the health research agenda.
Hmm, I wonder how well this would work?
The current insurance system is not a computational prediction market. You can not buy/sell mispriced securities, and thus you cannot profit from novel predictive information, and thus there is little to no incentive for insurance companies to fund research to solve health.
One of the main reasons that you can’t effectively do this are privacy protections. Government regulations that make the free market less efficient ;)
There are ways to handle privacy with crypto, but yeah I glossed overall that.
So would we have high-frequency-trading bots outside (or inside) of MRIs shorting the insurance policy value of people just diagnosed with cancer?
tl;dr: If the market does not already have an efficient mechanism for maximizing expected individual health (over all individuals who will ever live) then I take that as evidence that a complex derivative structure set up to purportedly achieve that goal more efficiently would instead be vulnerable to money-pumping.
Or to put a finer point on it; does the current market reward fixing and improving struggling companies or quickly driving them out of business and cutting them up for transplant into other companies?
Even further; does the current market value and strive to improve industries (companies aggregated by some measurement) that perform weakly relative to other industries? Or does the market tend to favor the growth of strong industries at the expense of the individual businesses making up the weak industries?
We’ve known how to cut health care costs and make it more efficient for centuries; let the weak/sick die and then take their stuff and give it to healthy/strong people.
I struggle to comprehend a free market that could simultaneously benefit all individual humans’ health while being driven by a profit motive. The free market has had centuries to come up with a way to reduce risk to individual businesses for the benefit of its shareholders; something that is highly desired by shareholders, who in fact make up the market. At best, investors can balance risk across companies and industries and hedge with complex financial instruments. Companies, however, buy insurance against uncertain outcomes that might reduce their value in the market. Sole proprietors are advised to form limited liability interests in their own companies purely to offset the personal financial risk. I can outlive Pentashagon LLC. but not my physical body as an investment vehicle that will be abandoned when it under-performs.
In short—yes—you want that information propagated through the market rapidly. It is the equivalent of credit assignment in learning systems. The market will learn to predict the outcome of the MRI—to the degree such a thing is possible.
Also, keep in mind that the insurance policy the patient holds is just one contract, and there could be layers of other financial contracts/bets in play untied to the policy (but correlated).
The proposal for using a computational market to solve health research has nothing whatsoever to do with wealth distribution. It obviously requires a government to protect the market mechanisms and enforce the rules, and is compatible with any amount of government subsidies or wealth redistribution. You seem to be conflating market mechanisms with political stances.
In theory a market can be used to solve any computational problem, provided one finds the right rules—this is the domain of computational mechanism design, an important branch of game theory.
That is possible, but the existing market has been under the reins of many a political stance and has basically obeyed the same general rules of economics, regardless of the political rules that have tried to be imposed.
The rules seem to be the weakest point of the system because they parallel the restrictions that political stances have caused to be placed on existing markets. If a computational market is coupled to the external world then it is probably possible to money-pump it against the spirit of the rules.
One way that a computational market could be unintentionally (and probably unavoidably) coupled to the external market is via status and signalling. Just like gold farmers in online games can sell virtual items to people with dollars, entities within the computational market could sell reputation or other results for real money in the external market. The U.S. FDA is an example of a rudimentary research market with rules that try to develop affordable, effective drugs. Pharmaceutical companies spend their money on advertising and patent wars instead of research. When the results of the computational market have economic effects in the wider market there will almost always be ways of gaming the system to win in the real world at the expense of optimizing the computation. In the worst case, the rule-makers themselves are subverted.
I am interested in concrete proposals to avoid those issues, but to me the problem sounds a lot like the longstanding problem of market regulation. How, specifically, will computational mechanism design succeed where years of social/economic/political trial and error have failed? I’m not particularly worried about coming up with game rules in which rational economic agents would solve a hard problem; I’m worried about embedding those game rules in a functioning micro-economy subject to interference from the outside world.
Oh—when I use the term “computational market”, I do not mean a market using fake money. I mean an algorithmic market using real money. Current financial markets are already somewhat computational, but they also have rather arbitrary restrictions and limitations that preclude much of the interesting computational space (such as generalized bet contracts ala prediction markets).
There is nothing inherently wrong with this or even obviously suboptimal about these behaviours. Advertising can be good and necessary when you have information which has high positive impact only when promoted—consider the case of smoking and cancer.
The general problem—as I discussed in the OP—is that the current market structure does not incentivize big pharma to solve health.
Well … yes.
Current political and economic structures are all essentially pre-information age technologies. There are many things which can only be done with big computers and the internet.
Also, I don’t see the years of trial and error so far as outright failures—it’s more of a mixed bag.
Now I realize that doesn’t specifically answer your question, but a really specific answer would involve a whole post or more.
But here’s a simple summary. It’s easier to start with the public single payer version of the idea rather than the private payer version.
The gov sets aside a budget—say 10 billion a year or so—for a health prediction market. They collect data from all the hospitals, clinics, etc and then aggregate and anonymize that data (with opt-in incentives for those who don’t care about anonymity). Anybody can download subsets of the data to train predictive models. There is an ongoing public competition—a market contest—where entrants attempt to predict various subsets of the new data before it is released (every month, week, day, whatever).
The best winning models are then used to predict the effect of possible interventions: what if demographic B3 was put on 2000 IU vit D? What if demographic Z2 stopped using coffee? What if demographic Y3 was put on drug ZB4? etc etc.
This allows the market to solve the hard prediction problems—by properly incentivizing the correct resource flow into individuals/companies that actually know what they are doing and have actual predictive ability. The gov then just mainly needs to decide roughly how much money these questions are worth.
What about predictions of the form “highly expensive and rare treatment F2 has marginal benefit at treating the common cold” that can drive a side market in selling F2 just to produce data for the competition? Especially if there are advertisements saying “Look at all these important/rich people betting that F2 helps to cure your cold” in which case the placebo affect will tend to bear out the prediction. What if tiny demographic G given treatment H2 is shorted against life expectancy by the doctors/nurses who are secretly administering H2.cyanide instead? There is already market pressure to distort reporting of drug prescriptions/administration and nonfavorable outcomes, not to mention outright insurance fraud. Adding more money will reinforce that behavior.
And how is the null prediction problem handled? I can predict pretty accurately that cohort X given sugar pills will have results very similar to the placebo affect. I can repeat that for sugar pill cohort X2, X3, …, XN and look like a really great predictor. It seems like judging the efficacy of tentative treatments is a prerequisite for judging the efficacy of predictors. Is there a theorem that shows it’s possible to distinguish useful predictors from useless predictors in most scenarios? Especially when allowing predictions over subsets of the data? I suppose one could not reward predictors who make vacuous predictions ex post facto, but that might have a chilling effect on predictors who would otherwise bet on homeopathy looking like a placebo.
Basically any sort of self-fulfilling prophesy looks like a way to steal money away from solving the health care problem.
You’ve misunderstood Jacob’s suggestion. Under his system there are no ‘claims’ - the health insurer simply pays for whatever healthcare it thinks will extend promote your health, up to the value it gets from your prolonged health (presumably around $100k / QALY )