Yes, that’s a sneaky part of the scenario. In general, I think this is a realistic thing to occur: ‘other intelligent people optimizing around this data’ is one of the things that causes the most complicated things to happen in real-world data as well.
Christian Z R had a very good comment on this, where they mentioned looking at the subset of dungeons where Rooms 2 and 4 had the same encounter, or where Rooms 6 and 8 had the same encounter, to factor out the impact of intelligence and guarantee ‘they will encounter this specific thing’.
(Edited to add: actually, there are ~100 rows in the dataset where Room2=4, Room6=8, and Room3=5=7. This isn’t enough to get firm analysis on, but it could have served as a very strong sanity-check opportunity where you can look at a few dungeons where you know exactly what the route is.)
actually, there are ~100 rows in the dataset where Room2=4, Room6=8, and Room3=5=7.
I actually did look at that (at least some subset with that property) at some point, though I didn’t (think of/ get around to) re-looking at it with my later understanding.
In general, I think this is a realistic thing to occur: ‘other intelligent people optimizing around this data’ is one of the things that causes the most complicated things to happen in real-world data as well.
Indeed, I am not complaining! It was a good, fair difficulty to deal with.
That being said, there was one aspect I did feel was probably more complicated than ideal, and that was the combination of the tier-dependent alerting with the tiers not having any other relevance than this one aspect. That is, if the alerting had in each case been simply dependent on whether the adventurers were coming from an empty room or not, it would have been a lot simpler to work out. And if there was tier dependent alerting, but the tiers were more obvious in other ways*, it would still have been tricky but at least there would be a path to recognize the tiers and then try to figure out other ways that they might have relevance. The way it was it seemed to me you pretty much had to look at what were (ex ante) almost arbitrary combinations of (current encounter, next encounter) to figure that aspect out, unless you actually guessed the rationale of the alerting effect.
That might be me rationalizing my failure to figure it out though!
* e.g. perhaps the traps/golems could have had the same score as the same-tier nontrap encounter when alerted (or alternatively when not alerted)
Mostly fair, but tiers did have a slight other impact in that they were used to bias the final room: Clay Golem and Hag were equally more-likely to be in the final room, both less so than Dragon and Steel Golem but more so than Orcs and Boulder Trap.
Yes, that’s a sneaky part of the scenario. In general, I think this is a realistic thing to occur: ‘other intelligent people optimizing around this data’ is one of the things that causes the most complicated things to happen in real-world data as well.
Christian Z R had a very good comment on this, where they mentioned looking at the subset of dungeons where Rooms 2 and 4 had the same encounter, or where Rooms 6 and 8 had the same encounter, to factor out the impact of intelligence and guarantee ‘they will encounter this specific thing’.
(Edited to add: actually, there are ~100 rows in the dataset where Room2=4, Room6=8, and Room3=5=7. This isn’t enough to get firm analysis on, but it could have served as a very strong sanity-check opportunity where you can look at a few dungeons where you know exactly what the route is.)
I actually did look at that (at least some subset with that property) at some point, though I didn’t (think of/ get around to) re-looking at it with my later understanding.
Indeed, I am not complaining! It was a good, fair difficulty to deal with.
That being said, there was one aspect I did feel was probably more complicated than ideal, and that was the combination of the tier-dependent alerting with the tiers not having any other relevance than this one aspect. That is, if the alerting had in each case been simply dependent on whether the adventurers were coming from an empty room or not, it would have been a lot simpler to work out. And if there was tier dependent alerting, but the tiers were more obvious in other ways*, it would still have been tricky but at least there would be a path to recognize the tiers and then try to figure out other ways that they might have relevance. The way it was it seemed to me you pretty much had to look at what were (ex ante) almost arbitrary combinations of (current encounter, next encounter) to figure that aspect out, unless you actually guessed the rationale of the alerting effect.
That might be me rationalizing my failure to figure it out though!
* e.g. perhaps the traps/golems could have had the same score as the same-tier nontrap encounter when alerted (or alternatively when not alerted)
Mostly fair, but tiers did have a slight other impact in that they were used to bias the final room: Clay Golem and Hag were equally more-likely to be in the final room, both less so than Dragon and Steel Golem but more so than Orcs and Boulder Trap.