Some initial rambling on what I’ve been doing before reading aphyer’s (and everyone else’s so far) comments; skip to TL;DR to avoid rambliness, I also request more time if possible:
This has been a fun problem so far. I expect that the inclusion of the Eldritch Abomination and Chaos Deity data are intended as Data Science Cheat Skills—though, the Eldritch Abomination so far doesn’t seem that useful, in contrast to the awesome Chaos Deity Cheat.
At the start, I thought to myself that the Eldritch Abomination data was probably intended to be used to learn about trait interactions in order to debias the no-collaborator data. (Though you could just use Chaos Deity data). I figured that in order for that to work, there had to be not much in the way of trait-skill interactions. But, in that case, the ordering of skill combos shouldn’t depend much on the traits, so the Chaos Deity data ignoring data should be about right.
So, I found the best skill combos according to Chaos Deity interactions and started by looking at what was best ignoring traits in the Chaos Deity data. I also noted some skill synergies and anti-synergies, noting that . Well, easy problem, now I just have to check if it works taking into account traits, then switch to just getting observations for the goddess right?
Wrong. I check all the data with the protagonist’s specific traits, and the skill combo relative win rates are very different.
But, now I think almost done though, because there’s so much data that even a single trait+skill combination has a decent number of data points, enough to be strongly suggestive of what’s best and while not enough to really be sure, but I’ll just have to check some “nearby” combos to see if it’s likely a fluke and then I’m done, right?
Wrong again. Because before I did that, first I wanted to just check to see if some hidden variables (which affect skill selection but also win rates) were affecting the result. So to do that I compared the results with Chaos Deity (for the protagonist’s traits) and they look—kind of different. Maybe not for sure different, but enough to be pretty suspicious.
So I look for more evidence of hidden variables, by (eventually) making a list of all the trait+skill combos and sorting by how unlikely[1] the chaos data (or a more extreme deviation from the Chaos+None data) would occur, assuming that the true distribution is given by the Chaos+None total data. And—holy crap, there is a freight train’s worth of evidence for hidden variables. Lots of outliers where specific skill combos show up consistently. In order to try to make sense of it, one theory I come up with classifies the skills into three groups—which I’ll discuss below. It is still seems flawed though, and I’m running out of time so I checked out the comments in this thread.
1 The formula I used was a cumulative binomial distribution with a conditional to determine which side I check. I am not confident this is the correct way to do it and it feels like it could easily be off by a factor of two or something (because some two-sided test might be more appropriate), but I guess it should give a reasonably accurate ordering of unlikeliness.
Further rambling about after reading the other comments:
I see aphyer’s made a detailed set of observations, and he and Yonge have both noted the Chaos Deity/no collaborator discrepancies to some extent.
What really excites me about aphyer’s comment though is a classification of the skills into three groups—Mental, Defense and Utility. Because these groups have a lot of similarity with the (flawed) classification I came up with from looking at discrepancies between the Chaos Deity and None data. (they also jive with my much more limited initial notes on synergies, which I had slipped out of mind).
My first group was Enlightenment/Radiant Splendor/Uncanny Luck, because for the trait/skill combos with the first 14 most unlikely discrepancies, both skills were from this group (and they all did better with no collaborator than with Chaos Deity). I called this these “light side” skills and, apart from the omission of Mind Palace, they correspond to aphyer’s “Mental” group.
My second group was Barrier Conjuration/Hypercompetent Dark Side/Monstrous Regeneration/Anomalous Agility. That was because, in the list of most unlikely discrepancies, up until spot 136 where these skills showed up they always did better with Chaos Deity than no collaborator. I called these “dark side” skills and, apart from the inclusion of Hypercompetent Dark Side, they match aphyer’s “Combat” (or “Defense”) group.
My third group was the remaining skills of Shapeshifting/Rapid XP Gain/Temporal Distortion/Mind Palace. I called these “neutral” skills and apart from the inclusion of Mind Palace and omission of Hypercompetent Dark Side, they match aphyer’s “Utility” group.
I had attempted to use these groups to predict which combos would do better with chaos and which with no collaborator. My working hypothesis had been: if the combination includes two “light side” skills, or a “light side” and a “neutral” skill, then there will be better results with no collaborator, otherwise there are better results with Chaos Deity. This correctly predicts which of the 82 most unlikely discrepancies favor Chaos Deity v. None. But, that may not be that great in terms of implications for correctness of the theory since some skills and skill combos are not showing up very much in the outliers extreme enough to be confident of non-randomness. For example, it’s not until the 99th most unlikely discrepancy that a dark+light combo shows up. And at this point there’s a decent fraction of a percent chance of this happening at random in any skill/trait combo making it very likely this will show up by chance.
One obvious question is, what happens if I modify my groups to match aphyer’s? Keeping the same rule, this fails on the 24th unlikeliest outlier, where Temporal Distortion + Mind Palace favors Chaos, but ought to favor None with the shift of Mind Palace. There are several other added failures in the first 100 or so, and all are attributable to Mind Palace. (And a number of these are not with the Hikkikomori trait). So, I think Mind Palace has to stay as I originally placed it, for this particular purpose of determining outliers).
Putting both “Mind Palace” and “Hypercompetent Dark Side” as “Utility”, the same as my original grouping but with HDS moved to utility: this rule makes the same decisions as the first up to the no. 177 point, so there is relatively weak evidence to distinguish which group HDS is in. It does look a little worse for this purpose to have it as “Utility”/”Neutral” than as “Defense”/”Dark Side”, but it could be either way. But the case against Mind Palace being in the “Light Side”/”Mental” group for this purpose seems relatively solid.
So anyway, I am trying to figure out more about this discrepancy information, but don’t have time. I would like to at least investigate the relationship to traits. So far all my analysis of it has no effect on my decision apart from confirming the existence of hidden variables. I also haven’t gotten around to looking at “nearby” trait combinations to look for trends/consistencies.
Anyway, as others have noted, it may be best to avoid a combo that did badly with the Chaos Deity, since that suggests that there’s some hidden variable that leads to selecting that combo and also to succeeding it. On the other hand, maybe these variables might not be hidden to the protagonist? If I feel a strong urge to select Enlightenment+Radiant Splendor, might as well go for it. I guess Temporal Distortion+Anomalous Agility otherwise, as the second best scorer for the protagonist’s exact traits.
Also, while the goddess’s reaction suggests that data analysis is unusual, I should probably ask just to verify. If not, that could change the implications of the hidden variables.
TL;DR:
There is an enormous amount of evidence of one or more hidden variables affecting both win rate and skill choice (as evidenced by discrepancies between the chaos deity and no-collaborator data for particular skill-and-trait combinations).
This includes both skill combinations that do better if you chose them yourself, and skill combinations that do worse.
This seems to be in some way connected to the groups that aphyer found, except that in this case Mind Palace seems to count as a “Utility” skill and maybe Hypercompetent Dark Side counts as a “Combat” skill (I use “Light”, “Dark” and “Neutral” instead of “Mental”, “Combat” and “Utility” for these altered groups) . Light being associated with doing better if you chose them yourself, and Dark (or double neutral, or maybe light+dark) doing worse. This rule is imperfect and the real rule could be completely different than what I think.
All of the above hasn’t actually had all that much practical impact yet. For now I pick:
Temporal Distortion + Anomalous Agility
(Same reasons as Yonge)
But I would pick Enlightenment + Radiant Splendor if that’s what I was going to pick without doing the analysis.
Some initial rambling on what I’ve been doing before reading aphyer’s (and everyone else’s so far) comments; skip to TL;DR to avoid rambliness, I also request more time if possible:
This has been a fun problem so far. I expect that the inclusion of the Eldritch Abomination and Chaos Deity data are intended as Data Science Cheat Skills—though, the Eldritch Abomination so far doesn’t seem that useful, in contrast to the awesome Chaos Deity Cheat.
At the start, I thought to myself that the Eldritch Abomination data was probably intended to be used to learn about trait interactions in order to debias the no-collaborator data. (Though you could just use Chaos Deity data). I figured that in order for that to work, there had to be not much in the way of trait-skill interactions. But, in that case, the ordering of skill combos shouldn’t depend much on the traits, so the Chaos Deity data ignoring data should be about right.
So, I found the best skill combos according to Chaos Deity interactions and started by looking at what was best ignoring traits in the Chaos Deity data. I also noted some skill synergies and anti-synergies, noting that . Well, easy problem, now I just have to check if it works taking into account traits, then switch to just getting observations for the goddess right?
Wrong. I check all the data with the protagonist’s specific traits, and the skill combo relative win rates are very different.
But, now I think almost done though, because there’s so much data that even a single trait+skill combination has a decent number of data points, enough to be strongly suggestive of what’s best and while not enough to really be sure, but I’ll just have to check some “nearby” combos to see if it’s likely a fluke and then I’m done, right?
Wrong again. Because before I did that, first I wanted to just check to see if some hidden variables (which affect skill selection but also win rates) were affecting the result. So to do that I compared the results with Chaos Deity (for the protagonist’s traits) and they look—kind of different. Maybe not for sure different, but enough to be pretty suspicious.
So I look for more evidence of hidden variables, by (eventually) making a list of all the trait+skill combos and sorting by how unlikely[1] the chaos data (or a more extreme deviation from the Chaos+None data) would occur, assuming that the true distribution is given by the Chaos+None total data. And—holy crap, there is a freight train’s worth of evidence for hidden variables. Lots of outliers where specific skill combos show up consistently. In order to try to make sense of it, one theory I come up with classifies the skills into three groups—which I’ll discuss below. It is still seems flawed though, and I’m running out of time so I checked out the comments in this thread.
1 The formula I used was a cumulative binomial distribution with a conditional to determine which side I check. I am not confident this is the correct way to do it and it feels like it could easily be off by a factor of two or something (because some two-sided test might be more appropriate), but I guess it should give a reasonably accurate ordering of unlikeliness.
Further rambling about after reading the other comments:
I see aphyer’s made a detailed set of observations, and he and Yonge have both noted the Chaos Deity/no collaborator discrepancies to some extent.
What really excites me about aphyer’s comment though is a classification of the skills into three groups—Mental, Defense and Utility. Because these groups have a lot of similarity with the (flawed) classification I came up with from looking at discrepancies between the Chaos Deity and None data. (they also jive with my much more limited initial notes on synergies, which I had slipped out of mind).
My first group was Enlightenment/Radiant Splendor/Uncanny Luck, because for the trait/skill combos with the first 14 most unlikely discrepancies, both skills were from this group (and they all did better with no collaborator than with Chaos Deity). I called this these “light side” skills and, apart from the omission of Mind Palace, they correspond to aphyer’s “Mental” group.
My second group was Barrier Conjuration/Hypercompetent Dark Side/Monstrous Regeneration/Anomalous Agility. That was because, in the list of most unlikely discrepancies, up until spot 136 where these skills showed up they always did better with Chaos Deity than no collaborator. I called these “dark side” skills and, apart from the inclusion of Hypercompetent Dark Side, they match aphyer’s “Combat” (or “Defense”) group.
My third group was the remaining skills of Shapeshifting/Rapid XP Gain/Temporal Distortion/Mind Palace. I called these “neutral” skills and apart from the inclusion of Mind Palace and omission of Hypercompetent Dark Side, they match aphyer’s “Utility” group.
I had attempted to use these groups to predict which combos would do better with chaos and which with no collaborator. My working hypothesis had been: if the combination includes two “light side” skills, or a “light side” and a “neutral” skill, then there will be better results with no collaborator, otherwise there are better results with Chaos Deity. This correctly predicts which of the 82 most unlikely discrepancies favor Chaos Deity v. None. But, that may not be that great in terms of implications for correctness of the theory since some skills and skill combos are not showing up very much in the outliers extreme enough to be confident of non-randomness. For example, it’s not until the 99th most unlikely discrepancy that a dark+light combo shows up. And at this point there’s a decent fraction of a percent chance of this happening at random in any skill/trait combo making it very likely this will show up by chance.
One obvious question is, what happens if I modify my groups to match aphyer’s? Keeping the same rule, this fails on the 24th unlikeliest outlier, where Temporal Distortion + Mind Palace favors Chaos, but ought to favor None with the shift of Mind Palace. There are several other added failures in the first 100 or so, and all are attributable to Mind Palace. (And a number of these are not with the Hikkikomori trait). So, I think Mind Palace has to stay as I originally placed it, for this particular purpose of determining outliers).
Putting both “Mind Palace” and “Hypercompetent Dark Side” as “Utility”, the same as my original grouping but with HDS moved to utility: this rule makes the same decisions as the first up to the no. 177 point, so there is relatively weak evidence to distinguish which group HDS is in. It does look a little worse for this purpose to have it as “Utility”/”Neutral” than as “Defense”/”Dark Side”, but it could be either way. But the case against Mind Palace being in the “Light Side”/”Mental” group for this purpose seems relatively solid.
So anyway, I am trying to figure out more about this discrepancy information, but don’t have time. I would like to at least investigate the relationship to traits. So far all my analysis of it has no effect on my decision apart from confirming the existence of hidden variables. I also haven’t gotten around to looking at “nearby” trait combinations to look for trends/consistencies.
Anyway, as others have noted, it may be best to avoid a combo that did badly with the Chaos Deity, since that suggests that there’s some hidden variable that leads to selecting that combo and also to succeeding it. On the other hand, maybe these variables might not be hidden to the protagonist? If I feel a strong urge to select Enlightenment+Radiant Splendor, might as well go for it. I guess Temporal Distortion+Anomalous Agility otherwise, as the second best scorer for the protagonist’s exact traits.
Also, while the goddess’s reaction suggests that data analysis is unusual, I should probably ask just to verify. If not, that could change the implications of the hidden variables.
TL;DR:
There is an enormous amount of evidence of one or more hidden variables affecting both win rate and skill choice (as evidenced by discrepancies between the chaos deity and no-collaborator data for particular skill-and-trait combinations).
This includes both skill combinations that do better if you chose them yourself, and skill combinations that do worse.
This seems to be in some way connected to the groups that aphyer found, except that in this case Mind Palace seems to count as a “Utility” skill and maybe Hypercompetent Dark Side counts as a “Combat” skill (I use “Light”, “Dark” and “Neutral” instead of “Mental”, “Combat” and “Utility” for these altered groups) . Light being associated with doing better if you chose them yourself, and Dark (or double neutral, or maybe light+dark) doing worse. This rule is imperfect and the real rule could be completely different than what I think.
All of the above hasn’t actually had all that much practical impact yet. For now I pick:
Temporal Distortion + Anomalous Agility
(Same reasons as Yonge)
But I would pick Enlightenment + Radiant Splendor if that’s what I was going to pick without doing the analysis.
Also, is it possible to add some time?
I would give you more time, but
you’ve already reached an optimal answer.
(Also, you can always just refuse to read the ruleset until you’re done with the data.)