What I found most interesting was people literally saying the words out loud, multiple times “Well, if this [assumption] isn’t true, then this is impossible” (often explicitly adding “I wouldn’t [normally] think this was that likely… but...”). And, then making the mental leap all the way towards “70% that this assumption is true.” Low enough for some plausible deniability, high enough to justify giving their plan a reasonable likelihood of success.
It was a much clearer instance of mentally slipping sideways into a more convenient world, than I’d have expected to get.
I think this tendency might’ve been exaggerated by the fact that they were working on a puzzle game; they know the levels are in fact solvable, so if a solution seems impossible it’s almost certainly the case that they’ve misunderstood something.
Yes, your prior on the puzzle being actually unsolvable should be very low, but in almost all such situations, 70% seems way too high a probability to assign to your first guess at what you’ve misunderstood.
When I prove to myself that a puzzle is impossible (given my beliefs about the rules), that normally leads to a period of desperate scrambling where I try lots of random unlikely crap just-in-case, and it’s rare (<10%) that anything I try during that desperate scramble actually works, let alone the first thing.
In the “final” level of Baba Is You, I was stuck for a long time with a precise detailed plan that solved everything in the level except that there was one step in the middle of the form “and then I magically get past this obstacle, somehow, even though it looks impossible.” I spent hours trying to solve that one obstacle. When I eventually beat the level, of course, it was not by solving that obstacle—it was by switching to a radically different approach that solved several other key problems in entirely different ways. In hindsight, I feel like I should have abandoned that earlier plan much sooner than I did.
In mitigation: I feel that solutions in Baba Is You are significantly harder to intuit than in most puzzle games.
Yes, but they failed with much higher than 30% likelihood even still. So the tendency you are pointing out wasn’t helping much. I do agree that it would be a better test to make multiple levels using the level editor, and have some of them literally be unsolvable. Then have someone have the choice of writing out a solution or writing out a proof for why there could be no solution. I think you’d find an even lower success rate in this harder version.
I think this tendency might’ve been exaggerated by the fact that they were working on a puzzle game; they know the levels are in fact solvable, so if a solution seems impossible it’s almost certainly the case that they’ve misunderstood something.
Yes, your prior on the puzzle being actually unsolvable should be very low, but in almost all such situations, 70% seems way too high a probability to assign to your first guess at what you’ve misunderstood.
When I prove to myself that a puzzle is impossible (given my beliefs about the rules), that normally leads to a period of desperate scrambling where I try lots of random unlikely crap just-in-case, and it’s rare (<10%) that anything I try during that desperate scramble actually works, let alone the first thing.
In the “final” level of Baba Is You, I was stuck for a long time with a precise detailed plan that solved everything in the level except that there was one step in the middle of the form “and then I magically get past this obstacle, somehow, even though it looks impossible.” I spent hours trying to solve that one obstacle. When I eventually beat the level, of course, it was not by solving that obstacle—it was by switching to a radically different approach that solved several other key problems in entirely different ways. In hindsight, I feel like I should have abandoned that earlier plan much sooner than I did.
In mitigation: I feel that solutions in Baba Is You are significantly harder to intuit than in most puzzle games.
Yes, but they failed with much higher than 30% likelihood even still. So the tendency you are pointing out wasn’t helping much. I do agree that it would be a better test to make multiple levels using the level editor, and have some of them literally be unsolvable. Then have someone have the choice of writing out a solution or writing out a proof for why there could be no solution. I think you’d find an even lower success rate in this harder version.