I’m pleasantly surprised by the effectiveness of my reasoning, and of my meta-reasoning. Not only did my loadout do well, but my calibration was impressively close: the final decision I pegged at a “~95%” success rate got 94.4%, and most of the alternative strategies I mentioned in my post were similarly on-the-nose.
(Unfortunately, my meta-meta-reasoning could still use some work. I figured out that this was a “linear-ish logistic success model with some interactions on top” kind of problem, took this as an opportunity to test that library I made, created a good predictor with a bunch of pretty/informative graphs . . . and then found myself thinking “only need one minigun? doesn’t sound right to me”, “why would Tyrants/Artillery and Scarabs/Minigun-or-Flamethrowers be so much stronger than every other potential feature interaction?”, and “I’m totally gonna turn out to have screwed up and wish I’d handled this with XGBoost, better not even mention how I built my model”. If I’d been more calibrated about how calibrated I ended up being, this could have been a really good chance to show off by calling in advance that my unconventional ML approach would succeed here.)
Reflections on the challenge:
This was the 2D performance thing I tried to pull off in Boojumologist, but with better conceptual underpinning and flawless execution. I’m proud, gladdened and envious: can’t think of a single way to improve this scenario.
(I, uh, may be biased by how well I happened to do: please take this feedback with a grain of salt.)
Reflections on my performance:
I’m pleasantly surprised by the effectiveness of my reasoning, and of my meta-reasoning. Not only did my loadout do well, but my calibration was impressively close: the final decision I pegged at a “~95%” success rate got 94.4%, and most of the alternative strategies I mentioned in my post were similarly on-the-nose.
(Unfortunately, my meta-meta-reasoning could still use some work. I figured out that this was a “linear-ish logistic success model with some interactions on top” kind of problem, took this as an opportunity to test that library I made, created a good predictor with a bunch of pretty/informative graphs . . . and then found myself thinking “only need one minigun? doesn’t sound right to me”, “why would Tyrants/Artillery and Scarabs/Minigun-or-Flamethrowers be so much stronger than every other potential feature interaction?”, and “I’m totally gonna turn out to have screwed up and wish I’d handled this with XGBoost, better not even mention how I built my model”. If I’d been more calibrated about how calibrated I ended up being, this could have been a really good chance to show off by calling in advance that my unconventional ML approach would succeed here.)
Reflections on the challenge:
This was the 2D performance thing I tried to pull off in Boojumologist, but with better conceptual underpinning and flawless execution. I’m proud, gladdened and envious: can’t think of a single way to improve this scenario.
(I, uh, may be biased by how well I happened to do: please take this feedback with a grain of salt.)