[EDITED 20h after posting because looking at someone else’s answer and being struck by how strongly it differed from mine despite doing nominally similar calculations prompted me to find a bug in my code.]
The given results include
a reasonable (but not enormous) number of results for every combination of (attacker,defender,territory) that can occur in our fights.
Accordingly, I don’t think there is any need to
look for patterns in the data in the hope of figuring out general rules; there’s enough noise that we shouldn’t be very confident about any patterns we find, so we might as well just use the data.
Specifically,
for each possible matchup I propose that we use Laplace’s rule of succession to estimate our winning probability should it happen; that is, if when A fights B in territory C we have so far seen A win a times and B win b times, we suppose that the probability of a win for A is (a+1)/(a+1+b+1). This corresponds to having a uniform prior on the winning probability. (Note in particular that when A has won every time we do not assume that this means A is literally unbeatable.)
We suppose
that different fights have independent outcomes; we have no obvious way to tell whether that’s so, still less to identify what specific correlations there might be, and on the face of it independence seems like a reasonable assumption in any case.
There are only
3840 different things we could do (120 permutations of the mages, times 32 choices of whether to attack or defend in each fight). I am assuming here that we have to fight all the battles, and that we can’t send more than one mage to a single battle. (If we decide to be helpful, then given that “as many wins as possible” is our summoner’s goal it seems clear that we should use all our mages, and we have no information about what happens if we use two at once, or what happens if an opponent has to attempt an attack immediately after successfully fighting off a counterattack.) So we can model all of them, see what distribution of #wins results from each, and then advise our summoner accordingly.
Of course this doesn’t determine
what we should actually say—because we might decide to be helpful, or to screw our summoner over in revenge for disturbing our rest, or to do some random trollish thing. I don’t really feel competent to legislate the utility functions of demons (maybe we love Evil and hate Good! maybe we love it when people do things that are bad for them! maybe we don’t care about the outcome of these fights and should “reward” our summoner for being naive, in the hope of screwing him over more vigorously on a later occasion! etc., etc., etc.), so I’ll offer some specific possibilities for specific goals we might have.
Specifically:
If we decide to be helpful by maximizing the expected number of wins for the “good” guys, they should send C,VB,GA,GB,VA respectively to the five upcoming fights; the cryomancer and geomancers should attack and the vitamancers defend. This way they get 4.24 wins on average. This is also the strategy that maximizes the chance of winning all five fights (40.9%), and the strategy that minimizes the chance of losing all five (5x10^-6).
If we decide to be malicious by minimizing the expected number of wins for the “good” guys, they should send VA,C,VB,GA,GB respectively to the five upcoming fights; the first two should attack, the others should defend. This way they get 0.69 wins on average. This is also the strategy that maximizes the chance of losing all five fights (46.4%). It isn’t the one that minimizes the chance of winning all five, which is obtained by swapping the two vitamancers and swapping the two geomancers; this makes the win-all-five probability 8x10^-6. (Of course the strategies where they almost never win all five, like the strategies where they almost never lose all five, are made that way by having at least one fight where we’ve never seen the wrong mage win, and the exact probability we end up with is strongly influenced by e.g. my choice to use Laplace’s rule of succession rather than something else.
If we decide to maximize our influence on the outcome by minimizing the variance of the number of wins for the “good” guys, they should send C,GB,VB,GA,VA respectively, the first three attacking and the other two defending. This way there’s an 84% chance that they win exactly one fight. This also minimizes the entropy of the outcome (sum of—p log p), if “outcome” means “number of wins”.
If we decide to maximize chaos by maximizing the variance of the number of wins for the “good” guys, they should send VB,VA,GB,C,GA to attack, attack, defend, defend, attack. This also maximizes the entropy of the outcome.
I expect
there are all sorts of interesting patterns in how the record of past wins and losses was generated; as explained above, I haven’t thought it worthwhile looking for them because I don’t think we can have enough evidence for favour such patterns over just using the results we have for outcomes given all the available information. I suppose it might turn out e.g. that territory makes no difference at all to the probabilities, in which case maybe we would get slightly better results by aggregating in the obvious way, but it’s not obvious that that’s so from a quick look at the data and I therefore don’t think it would be safe to assume it’s the case. (I mention this possibility because from the very quick and partial look I just took at the data, it’s also not 100% obvious that it’s false, though I see some things that look like evidence against.)
If I were spending more time on this, I would consider
models where e.g. there’s some territory-independent relationship between any two mages (and then a per-territory correction; more precisely I’d probably want something like log odds = a f1(attacker,defender) + b f2(attacker,territory) + c f3(defender,territory) + d f4(attacker,defender,territory); obviously we could always take a=b=c=0, but we’d put some sort of prior on the values of a,b,c,d, etc., etc., etc.; it all gets rather complicated), and try to estimate how strong that relationship is, which would give us the ability to use more information besides the results in matchups that correspond exactly to the ones we’re interested in—at a cost in worse results if it turns out that the model is a bad match for “reality”. I wouldn’t be terribly confident that this would actually do better than the naive approach I’ve taken above, but it would be interesting to play around with models of this sort and see how much if at all the results depend on what assumptions we make. Before doing any of that I’d obviously want to slice and dice the raw data in various ways in search of “obvious” patterns. And all this would take at least a couple of hours of extra work, and frankly I’m comfortable with the approach taken above which I expect to do reasonably well given any underlying reality that generates results that look broadly like the ones we have.
Ugh, I have a disastrous transcription error in my strategies above. I think what happened is that I copied the wrong output from my code—from an invocation where I had (as a result of a typo) asked to maximize the probability of getting exactly two wins (!). As a consequence, if the mages follow my “helpful” strategy they will do badly. One moral of this story is that anyone getting advice from me should ask me to double-check it.
Of course you have no way of confirming what I said in the previous paragraph, as opposed e.g. to the hypothesis that I brute-forced the interactive thingy that’s now been posted. (As it happens, the paragraph above is the truth. But you have no way of confirming that either. I did test out the strategy given above with the interactive thingy, which is how I noticed I’d screwed up.)
I’ve much less experience doing data analysis than @gjm appears to. This was a really nice way to get started, I may go back and try some of the other D&D exercises. My analysis was less mathematically rigorous, but the roster I got for being helpful seems close-ish to me. One thing I did note is that
there does seem to be a strong territorial advantage:
- Pyros: win most in pyro & vita territory - Vita: wins most in pyro and vita - Geo: wins most in geo and vita - Cryo: cryo, pyro - Necro: pyro, necro, cryo
My roster was more or less eyeballed by looking at stats of who seemed to loss most against whom and then selecting from the top two candidates. I got VA, VA, GA, GB, C; the first four all defending, C attacking.
I was toying with the idea of trying to figure out more complex analysis of this, but then got too curious and read gjm’s spoilers above, which made the point that there isn’t enough data to support serious modelling.
[EDITED 20h after posting because looking at someone else’s answer and being struck by how strongly it differed from mine despite doing nominally similar calculations prompted me to find a bug in my code.]
The given results include
a reasonable (but not enormous) number of results for every combination of (attacker,defender,territory) that can occur in our fights.
Accordingly, I don’t think there is any need to
look for patterns in the data in the hope of figuring out general rules; there’s enough noise that we shouldn’t be very confident about any patterns we find, so we might as well just use the data.
Specifically,
for each possible matchup I propose that we use Laplace’s rule of succession to estimate our winning probability should it happen; that is, if when A fights B in territory C we have so far seen A win a times and B win b times, we suppose that the probability of a win for A is (a+1)/(a+1+b+1). This corresponds to having a uniform prior on the winning probability. (Note in particular that when A has won every time we do not assume that this means A is literally unbeatable.)
We suppose
that different fights have independent outcomes; we have no obvious way to tell whether that’s so, still less to identify what specific correlations there might be, and on the face of it independence seems like a reasonable assumption in any case.
There are only
3840 different things we could do (120 permutations of the mages, times 32 choices of whether to attack or defend in each fight). I am assuming here that we have to fight all the battles, and that we can’t send more than one mage to a single battle. (If we decide to be helpful, then given that “as many wins as possible” is our summoner’s goal it seems clear that we should use all our mages, and we have no information about what happens if we use two at once, or what happens if an opponent has to attempt an attack immediately after successfully fighting off a counterattack.) So we can model all of them, see what distribution of #wins results from each, and then advise our summoner accordingly.
Of course this doesn’t determine
what we should actually say—because we might decide to be helpful, or to screw our summoner over in revenge for disturbing our rest, or to do some random trollish thing. I don’t really feel competent to legislate the utility functions of demons (maybe we love Evil and hate Good! maybe we love it when people do things that are bad for them! maybe we don’t care about the outcome of these fights and should “reward” our summoner for being naive, in the hope of screwing him over more vigorously on a later occasion! etc., etc., etc.), so I’ll offer some specific possibilities for specific goals we might have.
Specifically:
If we decide to be helpful by maximizing the expected number of wins for the “good” guys, they should send C,VB,GA,GB,VA respectively to the five upcoming fights; the cryomancer and geomancers should attack and the vitamancers defend. This way they get 4.24 wins on average. This is also the strategy that maximizes the chance of winning all five fights (40.9%), and the strategy that minimizes the chance of losing all five (5x10^-6).
If we decide to be malicious by minimizing the expected number of wins for the “good” guys, they should send VA,C,VB,GA,GB respectively to the five upcoming fights; the first two should attack, the others should defend. This way they get 0.69 wins on average. This is also the strategy that maximizes the chance of losing all five fights (46.4%). It isn’t the one that minimizes the chance of winning all five, which is obtained by swapping the two vitamancers and swapping the two geomancers; this makes the win-all-five probability 8x10^-6. (Of course the strategies where they almost never win all five, like the strategies where they almost never lose all five, are made that way by having at least one fight where we’ve never seen the wrong mage win, and the exact probability we end up with is strongly influenced by e.g. my choice to use Laplace’s rule of succession rather than something else.
If we decide to maximize our influence on the outcome by minimizing the variance of the number of wins for the “good” guys, they should send C,GB,VB,GA,VA respectively, the first three attacking and the other two defending. This way there’s an 84% chance that they win exactly one fight. This also minimizes the entropy of the outcome (sum of—p log p), if “outcome” means “number of wins”.
If we decide to maximize chaos by maximizing the variance of the number of wins for the “good” guys, they should send VB,VA,GB,C,GA to attack, attack, defend, defend, attack. This also maximizes the entropy of the outcome.
I expect
there are all sorts of interesting patterns in how the record of past wins and losses was generated; as explained above, I haven’t thought it worthwhile looking for them because I don’t think we can have enough evidence for favour such patterns over just using the results we have for outcomes given all the available information. I suppose it might turn out e.g. that territory makes no difference at all to the probabilities, in which case maybe we would get slightly better results by aggregating in the obvious way, but it’s not obvious that that’s so from a quick look at the data and I therefore don’t think it would be safe to assume it’s the case. (I mention this possibility because from the very quick and partial look I just took at the data, it’s also not 100% obvious that it’s false, though I see some things that look like evidence against.)
If I were spending more time on this, I would consider
models where e.g. there’s some territory-independent relationship between any two mages (and then a per-territory correction; more precisely I’d probably want something like log odds = a f1(attacker,defender) + b f2(attacker,territory) + c f3(defender,territory) + d f4(attacker,defender,territory); obviously we could always take a=b=c=0, but we’d put some sort of prior on the values of a,b,c,d, etc., etc., etc.; it all gets rather complicated), and try to estimate how strong that relationship is, which would give us the ability to use more information besides the results in matchups that correspond exactly to the ones we’re interested in—at a cost in worse results if it turns out that the model is a bad match for “reality”. I wouldn’t be terribly confident that this would actually do better than the naive approach I’ve taken above, but it would be interesting to play around with models of this sort and see how much if at all the results depend on what assumptions we make. Before doing any of that I’d obviously want to slice and dice the raw data in various ways in search of “obvious” patterns. And all this would take at least a couple of hours of extra work, and frankly I’m comfortable with the approach taken above which I expect to do reasonably well given any underlying reality that generates results that look broadly like the ones we have.
Ugh, I have a disastrous transcription error in my strategies above. I think what happened is that I copied the wrong output from my code—from an invocation where I had (as a result of a typo) asked to maximize the probability of getting exactly two wins (!). As a consequence, if the mages follow my “helpful” strategy they will do badly. One moral of this story is that anyone getting advice from me should ask me to double-check it.
Of course you have no way of confirming what I said in the previous paragraph, as opposed e.g. to the hypothesis that I brute-forced the interactive thingy that’s now been posted. (As it happens, the paragraph above is the truth. But you have no way of confirming that either. I did test out the strategy given above with the interactive thingy, which is how I noticed I’d screwed up.)
The helpful strategy I should have posted is
send C,VA,GA,VB,GB; geomancers defend, others attack.
D’oh!
I’ve much less experience doing data analysis than @gjm appears to. This was a really nice way to get started, I may go back and try some of the other D&D exercises. My analysis was less mathematically rigorous, but the roster I got for being helpful seems close-ish to me. One thing I did note is that
there does seem to be a strong territorial advantage:
- Pyros: win most in pyro & vita territory
- Vita: wins most in pyro and vita
- Geo: wins most in geo and vita
- Cryo: cryo, pyro
- Necro: pyro, necro, cryo
My roster was more or less eyeballed by looking at stats of who seemed to loss most against whom and then selecting from the top two candidates. I got VA, VA, GA, GB, C; the first four all defending, C attacking.
I was toying with the idea of trying to figure out more complex analysis of this, but then got too curious and read gjm’s spoilers above, which made the point that there isn’t enough data to support serious modelling.