Quantopian contest, but for food intake and weight
Twelve years ago, I lost 100 lbs. in a fairly boring manner by eating 1200 calories a day. Brutally unpleasant and catabolic, but mostly successful. It’s creeping back. There are no low-hanging fruits like sugarwater, fast food, eating out. The core problem is my satiety point is ~1000 calories above my RMR, and my body is not fooled by “high satiety” foods.
Since then, I’ve recorded every bite eaten every day, whether 10 calories of broth on a weekend fast or a 4000 calorie binge of my favorite deep dish “lucent” pizza, using a food scale for everything, not once thoughtlessly pouring oil into a recipe or blindly applying peanut butter to a sandwich. This cultivated superhuman calorie estimation abilities. “You say this is 280 kcal? It tastes like 350. Let me see the recipe… I see, the food label for ghee is off by half.” My goal in recording calories is to predict true fat loss by subtracting intake from RMR so daily scale variations aren’t discouraging. This is why I believe my records to be so much more accurate than others’—my incentives reward precision over the default of “undercount to stay under budget.” I weigh frequently, have impedance body composition data, periodic RMR data from an indirect calorimeter (which I now own), sleep-wake times, all gym sessions, and daily step counts.
Until last year, I believed CICO was cause rather than effect. For whatever reason, I thought, I’m hungrier or have less willpower, and so I must suffer to achieve a healthy weight. My beliefs have since turned toward a meandering set point controlled by a yet unknown homeostat. Whether contamination, omega ratios, a gravitostat, or some combination, anecdotal analysis suggests the set point is a hidden variable, confounding simple analysis. Weight loss when above it is trivial, and below it, fiendishly difficult. This is why I believe a control system is the most likely candidate for a successful model. Anything simpler would be a low-hanging fruit of simple correlation long ago ferreted out by nutrition science. It’s unlikely I’d be this size a century ago—almost no one was—so what is it I’m getting too much or not enough of?
Ideally, the massive longitudinal dataset contains dozens or hundreds of overlapping micro experiments (months of keto, weekend fasts, low protein, high fiber, potatoes, waves of monotonous meals) that taken together exceed the statistical significance of weeks spent in a lab testing individual hypotheses. Very likely, the answer is already in the data—a few weeks spread across years I gave up too early without realizing the noise in daily weigh-ins masked over-unity loss. If the answer to obesity requires a complex overlap or sequence of conditions, it may be hidden within and first discovered through data mining rather than invented whole cloth by a brilliant hypothesizer. You’d be hard pressed to find more or better data to mine.
I am 41, mostly retired, no prescriptions or real health problems, and no stressors or even major ups/downs. I have the budget and time to prepare food meeting any specifications, but cannot (out of squeamishness/disgust) consume animals (35 years) or eggs (~5 years). Dairy I’m still OK with, so don’t send any factory farm videos and ruin those. Not a big supplement fan unless it’s basic with a clear upside like magnesium or substituting for a missing (animal) nutrient like taurine.
Recalling the fun of Quantopian’s contest to write the best stock market prediction algorithm, I’d like to offer the same. Can your model, backtested on my 13 years of intake and weight data, predict what my weight would be given specific foods or macros? It won’t be a simple fit based on RMR. I’ve already built that; it drifts significantly over time, or weight loss would be a trivial exercise in using protein leverage to minimize total intake. It looks more like my set point moves up or down only when some conditions are met, and I’d like to find the control system at work. As an example, made of pure whimsy:
RMR_rolling = 2500
RMR_today = RMR_rolling
if (trailing_average(omega-6:3, 30 days) > 10:1) {
if (yesterday_protein > 50g)
RMR_today = RMR_rolling - 500
} else
RMR_today += min(max(intake - RMR_rolling, 0), 500)
if (steps_today * weight_current > 1,000,000 ft-lbs)
set_point -= 0.1 lbs
If it were that simple, maybe I could build or outsource it, but I’m looking for a statistician with the interest and creativity to devise control systems I can’t imagine.
As far as the bounty, a control system description and code to test it that doesn’t statistically overfit is worth at least $5,000. Diagnosing a subset of foods to exclude or increase, action to take and its precise value (3 mile walks shift set point down 4 ounces), or sustainable, non-miserable insight (only eat potatoes? tried thrice, data is in there) is worth $10,000. If I can put it into practice successfully for 6 months without suffering (highly subjective), and it moves my set point down >30 lbs., that’s worth a minimum of $20,000. If in backtesting it can predict today’s intake from current weight, set point, recent macros, etc., that’s phenomenal.
My fantasy? A model that asks: were you wearing a weighted vest on your summer 2020 walks, because the gravitostat underpredicted loss. Or, were you on vacation in a high altitude area this month? These bounties are minimums for mediocre, technically correct conclusions. For genuinely life-changing insight that allows me to live ad libitum, I’d go into six figures. I could just get a ’tide drug, but I’d first like to try for a victory on my factory settings that permanently pushes my set point down.
I am technically competent and will help in any way I can. If this kind of data analysis is your wheelhouse and such games are beneath you, I’ll simply meet your hourly rate. Same if you wish to work on a subset of the problem, like massaging the data into a database or extracting PUFA ratios.
Found this through a newsletter I subscribe to, this sounds like an interesting puzzle! I’m an economics and physics student semi-involved in machine learning, and while I probably can’t get to the level of detail you’re asking for, I’d be happy to throw the data through some adapted ML code (originally used for black hole mass calculations, but I think I could get it to work) in a few weeks over my winter break and send whatever output I get your way. Any results will be moderately imprecise (I suspect you would need a fair bit of computing power to get something actually specific given the size and number of variables in your dataset) but it sounds like a fun way to kill time and might even end up being slightly useful to you
Very interested in any unique approaches and I don’t want to constrain your creativity, but I am happy to jump on a 5-minute call to rapidly exchange ideas or discuss limitations or strengths of the data, if you think that might help. Also curious which newsletter so I can thank them for the signal boost.
I found this from Slime Mold Time Mold’s monthly links for Nov 2023. I imagine @underthesun did as well, but won’t speak for them.
I’m very interested in helping you solve your problem, but I am doubtful that it can be precisely solved with the data you have. Based on what I’ve read in the book Glucose Revolution (I am still critically evaluating its claims but so far appears to be solid), it seems that the body’s insulin response to a given food varies with pre-/post-intake exercise, whether a fibrous food was eaten beforehand, which types of other food were in the same meal, baseline levels of stress/sleep/insulin, and a host of other factors. Moreover, blood glucose spikes seem to lead to lots of poor mental and physical health outcomes (again, I’m still evaluating these claims).
Have you ever worn a continuous glucose monitor (CGM)? I suspect that some of your quandary may be solved by looking at high-frequency blood glucose readings. You may also be interested in ZOE which is a healthcare startup mentioned at the end of the book. ZOE advertises that it will monitor what seem to be the major measurable factors that affect insulin response: blood glucose, blood fat, and microbiome. And it claims to give personalized insights. The subscription price is much cheaper than the bounty you’re offering.
I’m not trying to promote Glucose Revolution or ZOE; just pointing out that blood sugar (as well as order-of-eating and other details) seems to be a key missing piece of your data. I applaud your desire to get the W on factory settings!
Feel free to reach out to me if you’d like to discuss offline; this is my same handle on twitter, Substack, GitHub, Gmail, etc. I will hope to write a Substack post soon looking deeper into the scientific backing of the book. I’m much more wary of these pop science books after Alexey Guzey destroyed Matthew Walker’s Why We Sleep.
This looks super interesting. I’ve done something similar for myself here. It would be interesting if similar results were found in your data.
Not what you wanted (but I can’t help you with that anyway), two things come to my mind:
What is your opinion on meals such as Soylent? It seems to me like something that should be easy to control with little suffering.
Have you tried semaglutide? I think the evidence shows that it works better than diets. Currently it is injections, but a pill version is being developed.
Added a sentence to the opener expressing the essential issue. Whether pure potatoes or meal replacement shakes, my body isn’t fooled by them. I have plenty of hunger to eat 2500 calories of potatoes or drink half a dozen shakes, so there’s no advantage over eating regular, interesting food.
If I can’t solve the mystery subtractively, a ’tide drug is the last resort. I don’t yet have a rationale I can articulate for wanting to succeed on my “factory settings” but the desire is there.
Ah, too bad. (To me it makes a difference: it is easy to overeat with normal food, but I have no such desire with the meal replacements.) By the way, have you tried adding edible fiber (psyllium) to your diet? That’s basically zero calories but helps fill the stomach.
I get it. Drugs often have side effects, and just having to buy and remember to eat them is inconvenient. It would be much nicer if things “just worked”. Sadly, often they don’t. :(
Given that you are mostly retired, maybe you could experiment more with energy expenditure? Like, try doing sport all day long for two weeks. Problem is, even if that solved the problem, it would probably take a lot of time, so you wouldn’t want to do this permanently. Is there a smart way to spend calories without spending time, for example always wearing a heavy backpack? Or just do some heavy-intensity exercise every morning and evening. Make your house colder. Put your computer on a walking desk. Etc.
You are drawing out many thoughts I had not explicitly stated in the post, so thank you. The contest is offered as such because I believe I’ve already stumbled into an answer and it is buried within the data. A couple weeks here and there when I ate certain macros or walked enough steps and prematurely quit, not realizing careful retrospective analysis reveals statistically significant loss above RMR for those periods that persisted for months, before being reversed by whatever combination of factors “unlocks” raising my set point.