abstractapplic

• abstractapplicOct 26, 2024, 7:20 PM

  4 points

  2

  on: A Differ­ent Per­spec­tive on Ra­tion­al­ity—Would This Be Valuable?

  I’m interested.

  (I’d offer more feedback, but that’s pretty difficult without an example to offer feedback on.)

The Sum­moned Heroine’s Pre­dic­tion Mar­kets Keep Pro­vid­ing Fi­nan­cial Ser­vices To The De­mon King!

• abstractapplicOct 24, 2024, 10:08 AM

  4 points

  0

  in reply to: simon’s comment on: D&D Sci Coli­seum: Arena of Data

  I tried fitting a model with only “Strength diff plus 8 times sign(speed diff)” as an explanatory variable, got (impressively, only moderately!) worse results. My best guess is that your model is underfitting, and over-attaching to the (good!) approximation you fed it, because it doesn’t have enough Total Learning to do anything better . . . in which case you might see different outcomes if you increased your number of trees and/​or your learning rate.

  Alternatively

  I might just have screwed up my code somehow.

  Still . . .

  I’m sticking with my choices for now.

• abstractapplicOct 21, 2024, 6:06 PM

  2 points

  0

  in reply to: abstractapplic’s comment on: D&D Sci Coli­seum: Arena of Data

  Update:

  I tried fitting my ML model without access to speed variables other than sign(speed diff) and got slightly but non-negligibly worse metrics on an outsample. This suggests that sign(speed diff) tells you most of the information you need about speed but if you rely solely on it you’re still missing useful and relevant information.

  (. . . either that or my code has another error, I guess. Looking forward to finding out in seven days.)

• abstractapplicOct 21, 2024, 4:26 PM

  4 points

  0

  in reply to: simon’s comment on: D&D Sci Coli­seum: Arena of Data

  Regarding my strategic approach

  I agree pick-characters-then-equipment has the limitation you describe—I’m still not sure about the B-vs-X matchup in particular—but I eyeballed some possible outcomes and they seem close enough to optimal that I’m not going to write any more code for this.

  I put your solution into my ML model and it seems to think

  That your A and C matchups are pretty good (though A could be made slightly better by benching Willow and letting Uzben do her job with the same gear), but B and D have <50% success odds.

  However

  I didn’t do much hyperparameter tuning and I’m working with a new model type, so it might have more epicycles than warranted.

  And

  “My model says the solution my model says is best is better than another solution” isn’t terribly reassuring.

  . . . regardless, I’m sticking with my choices.

  One last note:

  I don’t actually think there’s a strict +4 speed benefit cutoff—if I did I’d reallocate the +1 Boots from Y to V—but I suspect there’s some emergent property that kindasorta does the same thing in some highlevel fights maybe.

• abstractapplicOct 20, 2024, 5:48 AM

  2 points

  0

  in reply to: abstractapplic’s comment on: D&D Sci Coli­seum: Arena of Data

  Took an ML approach, got radically different results which I’m choosing to trust.

  Fit a LightGBM model to the raw data, and to the data transformed by simon’s stats-to-strength-and-speed model. Simon’s version got slightly better results on an outsample despite having many fewer degrees of freedom and fewer chances to ‘cheat’ by fingerprinting exceptional fighters; I therefore used that going forward. (I also tried tweaking some of the arbitrary constants in simon’s model: this invariably lowered performance, reassuring me that he got all the multipliers right.)

  Iterated all possible matchups, then all possible loadouts (modulo not using the +4 boots), looking for max EV of total count of wins.

  New strategy goes like this:

  Against A, send U, with +3 Boots

  Against B, send X, with +2 Boots and +1 Gauntlets

  Against C, send V, with +3 Gauntlets

  Against D, send Y, with +1 Boots and +2 Gauntlets

  Notes:

  The machines say this gives me ~2.6 expected victories but I’m selecting for things they liked so realistically I expect my EV somewhere in the 2-2.5 range.

  If I was doing this IRL I’d move the Gauntlets from V to U, lowering EV but (almost) guaranteeing me at least one win.

  My best guess about why my solution works (assuming it does) is that the “going faster than your opponent” bonus hits sharply diminishing returns around +4 speed. But that’s just post hoc confabulation.

• abstractapplicOct 20, 2024, 1:03 AM

  2 points

  0

  in reply to: simon’s comment on: D&D Sci Coli­seum: Arena of Data

  >only the last 12 having boots 2 and gauntlets 3 (likely post-theft)

  Didn’t notice that but it confirms my theory, nice.

  >It seems to me that they appear both as red and black, though.

  Ah, I see where the error in my code was that made me think otherwise. Strange coincidence: I thought “oh yeah a powerful wealthy elf ninja who pointedly wears black when assigned red clothes, what a neat but oddly specific 8-bit theater reference” and then it turned out to be a glitch.

• abstractapplicOct 19, 2024, 9:36 PM

  5 points

  0

  in reply to: simon’s comment on: D&D Sci Coli­seum: Arena of Data

  Noting that I read this (and that therefore you get partial credit for any solution I come up with from here on out): your model and the strategies it implies are both very interesting. I should be able to investigate them with ML alongside everything else, when/​if I get around to doing that.

  Regarding the Bonus Objective:

  I can’t figure out whether offering that guy we unknowingly robbed his shoes back is the best or the worst diplomatic approach our character could take, but yeah I’m pretty sure we both located the problem and roughly what it implies for the scenario.

• abstractapplicOct 19, 2024, 9:04 PM

  7 points

  0

  on: D&D Sci Coli­seum: Arena of Data

  I took an analytic approach and picked some reasonable choices based on that. I’ll almost certainly try throwing ML at this problem some point but for now I want to note down what a me-who-can’t-use-XGBoost would do.

  Findings:

  There are at least some fingerprintable gladiators who keep gladiating, and who need to be Accounted For (the presence of such people makes all archetypery suspect: are Dwarven Knights really that Good, or are there just a handful of super-prolific Dwarven Knights who give everyone an unfairly good impression?). This includes a Level 7 Elven Ninja, almost certainly Cadagal’s Champion, who inexplicably insists on always wearing black (even though it doesn’t seem to make a difference to how well ninjas ninj).

  Level 4 Boots and Level 4 Gauntlets are super rare in the dataset. The Gauntlets are always worn by a pair of hypercompetent Level 7 Dwarven Monks; the Boots are always worn by the Level 7 Elven Ninja.

  Despite this, Cadagal’s Champion is facing us with Level 2 Boots.

  We have some Level 4 Boots.

  . . . we robbed this guy, didn’t we? And if we wear the boots—our most powerful equipment—he’ll flip out and set his House against us whether we win or lose? Dammit . . .

  Who fights whom?

  A is a Human Warrior. Warriors lose to Fencers, Humans lose to Fencers, Humans lose to Elves. We have an Elven Fencer on call; send Y.

  B is a Human Knight. Rangers are best vs Knights, so send W. (Not super confident in this one)

  C is an Elven Ninja. Ninjas are super weak against Knights. Send Z, the Elven Knight. (Slightly concerned by how underrepresented Elves are in the sample of gladiators who managed to beat this guy but I’m assuming that’s either noise or an effect which Z will be able to shrug off with the Power of Friendship and/​or Urgency)

  D is a Dwarven Monk. Monks are weak to Ninjas; send U.

  Who wears what?

  I haven’t managed to figure out how equipment works beyond “higher number good”; if there’s specific synergies with/​against specific classes/​races/​whatever they elude me. For that reason:

  Y and Z are my best shots. I’ll have them both wear what their opponents are wearing, to reduce the effects of uncertainty and turn those fights into “who wore it better?” contests. (So +3 Boots and +1 Gauntlets for Y, +2 Boots and +3 Gauntlets for Z.)

  U vs D looks pretty solid so I’ll give him the remaining +2 Gauntlets and +1 Boots.

  W vs B is my most tenuous guess, I hope she won’t hold a grudge after I send her out unequipped to boost everyone else’s chances.

• abstractapplicOct 19, 2024, 12:13 PM

  23 points

  11

  on: What’s a good book for a tech­ni­cally-minded 11-year old?

  Math textbooks. Did you know that you can just buy math textbooks which are “several years too advanced for you”? And that due to economies of scale and the objectivity of their subject matter, they tend to be of both high and consistent quality? Not getting my parents to do this at that age is something I still regret decades later.

  Or did you specifically mean fiction? If so, you’re asking for fiction recommendations on the grew-up-reading-HPMOR website, we’re obviously going to recommend HPMOR (especially if they’ve already read Harry Potter, but it’s still good if you only know the broad strokes).

  What links here?

  • What’s a good book for a tech­ni­cally-minded 11-year old? by Martin Sustrik (Oct 19, 2024, 6:05 AM; 10 points)

• abstractapplicOct 17, 2024, 11:02 AM

  6 points

  3

  in reply to: Gregory ’s comment on: In­ter­est in Leet­code, but for Ra­tion­al­ity?

  Are you able to pinpoint exactly what gives you this feeling?

  Less a single sharp pinpoint, more a death of a thousand six cuts:

  • The emphasis on learning the names of biases is kinda guessing-the-teacher’s-password-y.

  • You’d need to put forth an unusual effort to make sure you’re communicating the subset of psychological research which actually replicates reliably.

  • Any given bias might not be present in the student or their social/​business circle.

  • The suggested approach implies that the set of joints psychologists currently carve at is the ‘best’ one; what if I happen to see Bias A and Bias B as manifestations of Bias C?

  • I worry some students would round this off to “here’s how to pathologize people who disagree with me!” training.

  • Like I said, this is the kind of fruit that’s low-hanging enough that it’s mostly already picked.

  All that said, I still think this is potentially worthwhile and would still playtest it if you wanted. But I’m much more excited about literally every other idea you mentioned.

• abstractapplicOct 16, 2024, 9:56 PM

  36 points

  3

  on: In­ter­est in Leet­code, but for Ra­tion­al­ity?

  I am extremely interested in this, and all similar efforts in this space. I agree our community should be doing much more along these lines.

  Regarding your specific ideas:

  Cognitive Bias Detection

  Something about training people to categorize errors—instead of just making good decisions—rubs me the wrong way. Also, there’s a lot of pre-existing work (I found out about this earlier today).

  Calibration Training

  The Credence Calibration Game exists. So does my variation on the same idea (see also the associated lesson plan). So do play-money and real-money prediction markets. That said, I do think there’s a valuable and unfilled niche for something which doesn’t require a download and has a nice user interface and has a four-digit number of questions and lets you check your answers immediately (. . . though I don’t know how many people other than me would consider it valuable).

  Bite-Sized, Practical Challenges

  I am very much in favor of this, to the point where I’m already (tentatively) planning to (eventually) build some games with a similar motivation. Relatedly, the “ask users to predict an outcome based on limited data” example sounds like a description of that genre I invented (though “Bite-Sized” suggests you’re thinking in terms of something much more polished/​generally-accessible).

  (Side note: A subtle benefit of the “Practical Challenges” approach is that it can correct for biases you weren’t aiming for. A large part of my motivation for making D&D.Sci was “forcing them to confront the common pitfalls of overconfidence or representativeness heuristics”; I found that a Lesswronger working in a Data Science context will more often be insufficiently confident, and place too little weight on surface appearances; my endeavor ‘failed’ gracefully and people got a chance to notice those errors instead (plus various other problems I didn’t even consider).)

  -

  I look forward to seeing what comes of this. If you want anything playtested, please let me know.

• abstractapplicOct 11, 2024, 10:20 PM

  7 points

  2

  on: Open let­ter to young EAs

  As with the last article, I think this is almost entirely incoherent/​wrong; as with the last article, I’m strong-upvoting it anyway because I think it makes ~1.5 good and important points I’ve not seen made anywhere else, and they’re worth the chaff. (I’d go into more detail but I don’t want anyone leaning on my summary instead of reading it themselves.)

  . . . is there a reason this is a link to a google doc, instead of a copypaste?

  

  ^[1]

  1. ^

     and add footnotes.

• abstractapplicOct 3, 2024, 10:23 PM

  14 points

  2

  in reply to: Robert Miles’s comment on: Three Sub­tle Ex­am­ples of Data Leakage

  “Using information during Training and/​or Evaluation of models which wouldn’t be available in Deployment.”

  . . . I’ll edit that into the start of the post.

Three Sub­tle Ex­am­ples of Data Leakage

• abstractapplicSep 27, 2024, 5:16 PM

  3 points

  0

  in reply to: Christian Z R’s comment on: D&D.Sci: Whom Shall You Call? [Eval­u­a­tion and Rule­set]

  Thanks for a good one

  I’m glad you feel that way about this scenario. I wish I did . . .

  (For future reference, on the off-chance you haven’t seen it: there’s a compilation of all the past scenarios here, handily rated by quality and steamrollability.)

  One thing that perhaps would make it easier was if the web interactive could tell whether or not your selection was the optimal one directly, and possibly how higher your expected price was than the optimal price (I first plugged mine in, then had to double check with your table out here)

  . . . huh. I feel conflicted about this on aesthetic grounds—like, Reality doesn’t come with big flashing signs saying “EV-maxxing solution reached!” when you reach an EV-maxxing solution—but it does sound both convenient to have and easy to set up. Might try adding this functionality to the interactive for the next one; would be curious to hear what anyone else who happens to be reading this comment thinks.

  Anyway, greetings, and looking forward to seeing the next one.

  Good to have you on board!

• abstractapplicSep 18, 2024, 1:17 PM

  6 points

  0

  in reply to: Joseph Miller’s comment on: I fi­nally got ChatGPT to sound like me

  I really wish I could simultaneously strong-upvote and strong-downvote the “agree” thing for this reply. I think most of the description is horoscope-y flattery, but it doesn’t have zero correlation with reality: I do think lsusr’s writing is ninety-something-th percentile for

  a kind of minimalist clarity that leaves room for the reader to reflect and draw their own conclusions

  and at least eighty-something-th percentile for

  willing to question common assumptions within the rationalist sphere

  while afaict there’s nothing in the description that’s the opposite of the truth.

  (I also think there’s something interesting about how the most meaningful/​distinguishing lines in the description are the ones which could be most easily rephrased as criticisms. Does “describe X as neutrally as possible” or “describe the upsides and downsides of X” produce better LLM results than “describe X”?)

• abstractapplicSep 15, 2024, 4:44 PM

  2 points

  6

  on: ab­strac­tap­plic’s Shortform

  I used to implicitly believe that when I have a new idea for a creative(/​creative-adjacent) project, all else being equal, I should add it to the end of my to-do list (FIFO). I now explicitly believe the opposite: that the fresher an idea is, the sooner I should get started making it a reality (LIFO). This way:

  • I get to use the burst of inspired-by-a-new-idea energy on the project in question.

  • I spend more time working on projects conceived by a me with whom I have a lot in common.

  The downsides are:

  • Some old ideas will end up near the bottom of the pile until I die or the Singularity happens. (But concepts are cheaper than execution, and time is finite.)

  • I get less time to polish ideas in my head before committing pen to paper. (But maybe that’s good?)

  Thoughts on this would be appreciated.

ab­strac­tap­plic’s Shortform

• abstractapplicAug 29, 2024, 11:36 AM

  3 points

  0

  in reply to: pzas’s comment on: Devel­op­ing Pos­i­tive Habits through Video Games

  Something low complexity, repeatable, non-addictive, non-time consuming.

  I think a lot of people have a habit like that, and it’s different things for different people.

  I think it’s meditation

  Beware.

  (Like, from what I hear you’re not wrong, but . . . y’know, Beware.)