D&D.Sci Alchemy: Archmage Anachronos and the Supply Chain Issues Evaluation & Ruleset
This is a follow-up to last week’s D&D.Sci scenario: if you intend to play that, and haven’t done so yet, you should do so now before spoiling yourself.
There is a web interactive here you can use to test your answer, and generation code available here if you’re interested, or you can read on for the ruleset and scores.
RULESET
There are two steps to brewing a potion:
STEP 1: MAGICAL POTENCY
Any ingredient that doesn’t exist in the mundane world is Magical, while any ingredient that exists in the mundane world is not:
| Magical | Not Magical |
| Angel Feather | Badger Skull |
| Beholder Eye | Beech Bark |
| Demon Claw | Crushed Diamond |
| Dragon Scale | Crushed Onyx |
| Dragon Spleen | Crushed Ruby |
| Dragon Tongue | Crushed Sapphire |
| Dragon’s Blood | Eye of Newt |
| Ectoplasm | Ground Bone |
| Faerie Tears | Oaken Twigs |
| Giant’s Toe | Powdered Silver |
| Troll Blood | Quicksilver |
| Vampire Fang | Redwood Sap |
The first step of potion-brewing is to dissolve the magical potency out of the Magical Ingredients to empower your potion. This requires the right amount of Magical Ingredients: too few, and nothing magical will happen and you will produce Inert Glop, while too many and there will be an uncontrolled Magical Explosion.
If you include:
0-1 Magical Ingredients: 100% chance of Inert Glop.
2 Magical Ingredients: 50% chance of Inert Glop, 50% chance OK.
3 Magical Ingredients: 100% chance OK.
4 Magical Ingredients: 50% chance OK, 50% chance Magical Explosion.
5+ Magical Ingredients: 100% chance Magical Explosion.
If your potion got past this step OK, move on to:
STEP 2: DIRECTION
Some ingredients are used to direct the magical power into the desired resulting potion. Each potion has two required Key Ingredients, both of which must be included to make it:
| Potion | Key Ingredient 1 | Key Ingredient 2 |
| Barkskin Potion* | Crushed Onyx | Ground Bone |
| Farsight Potion | Beholder Eye | Eye of Newt |
| Fire Breathing Potion | Dragon Spleen | Dragon’s Blood |
| Fire Resist Potion | Crushed Ruby | Dragon Scale |
| Glibness Potion | Dragon Tongue | Powdered Silver |
| Growth Potion | Giant’s Toe | Redwood Sap |
| Invisibility Potion | Crushed Diamond | Ectoplasm |
| Necromantic Power Potion* | Beech Bark | Oaken Twigs |
| Rage Potion | Badger Skull | Demon Claw |
| Regeneration Potion | Troll Blood | Vampire Fang |
*Well. Sort of. See the Bonus Objective section below.
Some ingredients (Angel Feather, Crushed Sapphire, Faerie Tears and Quicksilver) aren’t Key Ingredients for any potion in the dataset. Angel Feather and Faerie Tears are nevertheless useful—as magical ingredients that don’t risk creating any clashing potion, they’re good ways to add magical potential to a recipe. Crushed Sapphire and Quicksilver have no effect, including them is entirely wasteful.
If you’ve gotten through Step 1, the outcome depends on how many potions you’ve included both the Key Ingredients of:
0 potions: with nothing to direct it, the magical potential dissolves into an Acidic Slurry.
1 potion: you successfully produce that potion.
2 or more potions:
Sometimes (1/n of the time, where n is # of potions you included) a random one of the potions will dominate, and you will produce that one.
The rest of the time, the clashing directions will produce Mutagenic Ooze.
So, for example, if you brew a potion with:
Dragon Spleen, Dragon Scale, Dragon Tongue and Dragon’s Blood:
You have included 4 magical ingredients, and the Key Ingredients of one potion (Fire Breathing).
50% of the time you will get a Magical Explosion, 50% of the time you will get a Fire Breathing Potion.
Badger Skull, Demon Claw, Giant’s Toe, Redwood Sap.
You have included 2 magical ingredients, and the Key Ingredients of two potions (Rage and Growth).
50% of the time you will get Inert Glop, 25% of the time Mutagenic Ooze, 12.5% of the time Growth Potion, and 12.5% of the time Rage Potion.
Dragon’s Tongue, Powdered Silver, Angel Feather and Faerie Tears:
You have included 3 magical ingredients, and the Key Ingredients of one potion (Glibness).
100% of the time, you will successfully brew a Glibness Potion.
STRATEGY
For best success, you needed to:
Include Crushed Onyx and Ground Bone, the Key Ingredients for...the desired potion.
Not include both Key Ingredients of any other potion.
Include exactly 3 Magical Ingredients.
This was made trickier by the fact that you had only 4 Magical Ingredients available (Demon Claw, Giant’s Toe, Troll Blood and Vampire Fang), and that Troll Blood and Vampire Fang were the Key Ingredients of a Regeneration Potion (and hence you could not include both of them).
Nevertheless, you could get a 100% success rate by:
Including Crushed Onyx and Ground Bone (Key Ingredients).
Including Demon Claw and Giant’s Toe (Magical Ingredients that don’t make another potion with one another)
Including one of Troll Blood and Vampire Fang (third Magical Ingredient).
Not including Badger Skull (makes Rage Potion with Demon Claw) or Redwood Sap (makes Growth Potion with Giant’s Toe).
Not including both Beech Bark and Oaken Twigs (Key Ingredients of...the wrong potion).
You could include Crushed Diamond or Quicksilver if you wanted, but it’s wasteful and has no effect.
BONUS OBJECTIVE
There was a secret bonus objective hidden in the dataset. It could not be found purely with math on the dataset, it required you to think about the ingredient names and the patterns you were seeing.
On examination of the Key Ingredients of each potion, most potions had Key Ingredients that seemed in some way vaguely metaphorically linked to the potion’s effect:
The Key Ingredients of a Farsight Potion are Beholder Eye and Eye of Newt (both are eyes, to help you see).
The Key Ingredients of a Fire Breathing Potion are Dragon Spleen and Dragon’s Blood (both parts of a fire-breathing dragon).
The Key Ingredients of a Growth Potion are Giant’s Toe and Redwood Sap (both from things well known for growing large).
There were two apparent exceptions to this, however.
The Key Ingredients of a Barkskin Potion appeared to be Crushed Onyx and Ground Bone.
The Key Ingredients of a Necromantic Power Potion appeared to be Beech Bark and Oaken Twigs.
This was not, in fact, true.
The Key Ingredients of a Barkskin Potion are in fact Beech Bark and Oaken Twigs, and the Key Ingredients of a Necromantic Power Potion are Crushed Onyx and Ground Bone.
Archmage Anachronos is not trying to brew a Barkskin Potion. He is trying to brew a Necromantic Power Potion in order to conduct a Dread Necromantic Ritual during the upcoming magical conjunction.
Not wanting to reveal this, he’s swapped around ‘Barkskin Potion’ and ‘Necromantic Power Potion’ in his records of results, telling you that every recipe that actually produced a Barkskin Potion produced a Necromantic Power Potion and vice versa.
simon realized this, I don’t think anyone else figured it out independently (as opposed to seeing his comment mentioning it).
DATASET GENERATION
Archmage Anachronos’s algorithm for deciding which ingredients to use is as follows:
20% of brews are experiments. He grabs 1d6+2 random ingredients to test together, but if they seem too expensive he reconsiders and grabs a different 1d6+2 ingredients (causing a bias in his experiments towards both smaller numbers of ingredients and cheaper ingredients).
80% of brews are attempts to make a specific potion. He chooses a random potion to try to produce. Then he picks a random recipe that he has successfully made that potion with in the past, and repeats that recipe. (He does not have a good enough recollection to try a recipe that’s had high odds of success at brewing that potion in the past, he’ll try any recipe that has ever brewed that potion).
The main relevance of this algorithm to players was that some combinations of ingredients, especially ones that could produce potions and especially ones that could produce multiple different potions, had been brewed many more times than would happen under pure randomness.
LEADERBOARD
This scenario was intended to be relatively straightforward, and bearing this out we had several perfect scores:
simon fully deciphered the algorithm, got a 100% success rate and also solved the Bonus Objective.
abstractapplic and Lorxus also both got a 100% success rate at brewing ‘Barkskin Potion’, but did not independently solve the Bonus Objective until they saw simon’s solution to it.
Unnamed, Yonge, and qwertyasdef all got a 50% success rate:
Yonge and Unnamed looked at past brewing and tried Crushed Onyx, Demon Claw, Ground Bone and Vampire Fang (50% to succeed, but low on magic).
qwertyasdef instead used Crushed Onyx, Demon Claw, Ground Bone, Quicksilver, and Troll Blood based on an analysis of which available ingredients often appeared in ‘Barkskin Potion’, but had the same result of 50% success due to low magic.
Congratulations to all players! Particular congratulations to simon for figuring out the Bonus Objective! Maniacal cackling to all other players!
REFLECTION & FEEDBACK REQUEST
The goal I was shooting for with this scenario was to reward people for actually paying human attention to patterns, rather than just hoping for [YOUR FAVORITE ALGORITHM HERE] to solve all your problems for you.
The Bonus Objective was obviously written to require out-of-context thought. I also tried to set up the scenario to be amenable to thinking about what patterns might mean even with relatively little math: bearing this out, it looks like Lorxus managed to get a perfect score with relatively little in the way of complicated methods/tools just by e.g. thinking about what it might mean that including lots of ingredients led to Magical Explosions and including few ingredients led to Inert Glop (and documented his thought process very well, thank you Lorxus!).
How did this feel from the player end?
As usual, I’m also interested to hear more general feedback on what people thought of this scenario. If you played it, what did you like and what did you not like? If you might have played it but decided not to, what drove you away? What would you like to see more of/less of in future? Do you think the scenario was more complicated than you would have liked? Or too simple to have anything interesting/realistic to uncover? Or both at once? Do you have any other feedback?
I think this is plausibly the best scenario either of us have made to date.
The basic game was very good, layering simple rules on top of each other to create a complex system which was challenging to detangle but easy to understand once you know the rules. I was particularly impressed by the fact that you managed the (imo, near-impossible) feat of making an enjoyable D&D.Sci where handling data starvation was a key part of the problem: most players (including me) seem to have had the (sensible) thought “okay, let’s filter for only potions with Onyx and Bone”, and success past that point was predicated on realizing there weren’t quite enough rows to justify being that picky.
The twist struck me as fair, funny and fun. It provided an object lesson in noticing when things don’t quite add up, and letting the qualitative shade the quantitative; it also expanded the scope of the genre in ways I realize I’ve been neglecting to.
All that said, I have some (minor, petty) criticisms . . . not of the game itself, but how it was presented. Namely:
.This entry was billed as “relatively simple”, but I think it was about median difficulty by the standards of D&D.Sci; pretty sure it was harder than (for example) The Sorceror’s Personal Shopper.
.”STORY (skippable)” was kind of misleading this time: the flavortext had a lot of little hints that the Archmage wasn’t on the level, so someone who didn’t read it (or failed to read between the lines, like me) would be at a (small) disadvantage.
.”Archmage Anachronos is trying to brew Barkskin Potion” was A) the GM saying something false directly to the players, and B) a missed opportunity: if you’d written something like “Your goal is to help Archmage Anachronos brew Barkskin Potion”, that would have been a subtle confirmation that giving him exactly what he asked for would lead to the best outcome (vs more aggressive / galaxy-brained forms of sabotage, or refusing to cast judgement on his pursuit of immortality, or any other reaction).
I’m glad you liked it, thank you!
I guess that’s fair. There’s a complication here in that...uh...almost all of my scenarios have been above-median complexity and almost all of yours have been below-median. (I should probably write down my thoughts on this at some point). I agree that this one wasn’t simpler than most of yours, but I think that it was still a much more approachable entry point than e.g. Duels & D.Sci, or League of Defenders.
(It’s possible we should try to standardize a 1-10 complexity scale or some such so that we can stick a difficulty rating on the top of each scenario.)
Fair enough, I can tweak that for anyone who finds the scenario in future.
I intended that the story should not provide much help...the intent was not for players to notice that Anachronos was suspicious in-story, the intent was for them to notice from the data, and for the hints in the story to be just some quiet confirmation for a player who realized the twist from the data and then went back to reread the story.
On the other hand, I was expecting more players to get the twist, and thought that I’d only really catch players who ignored the ingredient names entirely and just fed the data into an ML algorithm, so I’m clearly not very well calibrated on this. I was really quite surprised by how many players analyzed the data well enough to say “Barkskin potion requires Crushed Onyx and Ground Bone, Necromantic Power Potion requires Beech Bark and Oaken Twigs” and then went on to say “this sounds reasonable, I have no further questions.” (Maybe the onyx-necromancy connection is more D&D lore than most players knew? But I thought that the bone-necromancy and bark-barkskin connections would be obvious even without that).
I...think I’m in general allowed to say false things directly to the players as a D&D GM? If the Big Bad is disguised as your innkeeper while the real innkeeper is tied up in the cellar, I think I can say ‘The innkeeper tells you it’ll be six silver for a room’, I don’t think I need to say ‘The man who introduced himself to you as the innkeeper.’
(Also, you are a Data Scientist. Sense Motive is not a class skill for you. Clearly you failed a Sense Motive check and so believed him!)
...I’ll think about whether I want to tweak that line for potential future players.
I thought the flavour text was just right—I got it from the data, not the flavour text, and saw the flavour text as confirmation, as you intended.
Illusion of transparency I think, hints are harder than anyone making them thinks.
When I looked at the ingredients for a “barkskin potion”, as far as I knew at this point the ingredients were arbitrary, so in fact I don’t recall finding it suspicious at all. Then later I remember looking at the ingredients for a “necromantic power potion” and thinking something like… “uh… maybe wood stuff is used for wands or something to do necromancy?”. It was only when I explicitly made a list of the ingredients for each potion type, rather than looking at each potion individually, and could see that everything else make sense, that I realized the twist.
Perhaps, but you could also simply say “Yeah, the guy at the counter tells you the room will be 6 silver.”
Not quite true!That’s where I started to break through, but after that I noticed the Mutagenic Ooze issue as well. It also took me a lot of very careful graceful use of pivot tables. Gods beyond, that table chugged. (And if I can pull the same Truth from the void with less powerful tools, should that not mark me as more powerous in the Art? :P)I guess I’m not clear on what “actual Data Science” would involve, if not making hypotheses and then conducting observational-experiments?I figured out the MO mechanic specifically by looking at brews that coded for pairs of potions, for the major example. The only thing that would have changed if I’d known SQL would be speed, I suspect.Always a pleasure! I had a lot of fun with this one. I was a little annoyed by the undeclared bonus objective—I would have wanted any indication at all in the problem statement that anything was not as it appeared. I did notice the correspondence in (i.a.) the Farsight Potion but in the absence of any reason to suspect that the names were anything but fluff, I abstracted away anything past the ingredients being a set of names. Maybe be minimally more obvious? At any rate I’d be happy to be just as detailed in future, if that’s something you want.
I liked the bonus objective myself, but maybe I’m biased about that...
As a someone who is also not a “data scientist” (but just plays one on lesswrong), I also don’t know what exactly actual “data science” is, but I guess it’s likely intended to mean using more advanced techniques?
Perhaps, but don’t make a virtue of not using the more powerful tools, the objective is to find the truth, not to find it with handicaps...
Speaking of which one thing that could help making things easier is aggregating data, eliminating information you think is irrelevant. For example, in this case, I assumed early on (without actually checking) that timing would likely be irrelevant, so aggregated data for ingredient combinations. As in, each tried ingredient combination gets only one row, with the numbers of different outcomes listed. You can do this by assigning a unique identifier to each ingredient combination (in this case you can just concatenate over the ingredient list), then counting the results for the different unique identifiers. Countifs has poor performance for large data sets, but you can sort using the identifiers then make a column that adds up the number of rows (or, the number of rows with a particular outcome) since the last change in the identifier, and then filter the rows for the last row before the change in the identifier (be wary of off-by-one errors). Then copy the result (values only) to a new sheet.
This also reduces the number of rows, though not enormously in this case.
Of course, in this case, it turns out that timing was relevant, not for outcomes but only for the ingredient selection (so I would have had to reconsider this assumption to figure out the ingredient selection).
I’m obviously seeking out more powerful tools, too—I just haven’t got them yet. I don’t think it’s intrinsically good to stick to less powerful tools, but I do think that it’s intrinsically good to be able to fall back to those tools if you can still win.
And when I need to go out and find truth for real, I don’t deny myself tools, and I rarely go it alone. But this is not that.
You don’t need to justify—hail fellow D&Dsci player, I appreciate your competition and detailed writeup of your results, and I hope to see you in the next d&dsci!
I have struckthrough part of the previous comment, given the edit. I need no longer stand by it as a complaint.