Thinking about how exactly design stories help create trust, I came upon what might be a useful distinction: whether the design is good according to the considerations known to the designer, vs. whether all relevant considerations are present. A good design story lets us check both of these. The first being false means the designer just did a bad job, or perhaps is hiding something. The second being false means there are actually just considerations the designer didn’t know about—for example because they live implicit in some other human’s head—and spelling things out in a story lets us recognize that, and correct it.
The latter use of stories lets you catch honest mistakes around issues that are unknown unknowns to you, but knowns for someone else. And when I think intuitively about trusting an AI—or another human for that matter—this is a big part of what I care about: beyond them being competent, and not actively deceiving me, I should also trust that they’ll communicate with me enough to fill in all the blind spots they might have about me and the things I care about.
Yes when we have a shared understanding of what we’re building together, with honest and concise stories flowing in both directions, we have a better chance of actually understanding what all the stakeholders are trying to achieve, which at least makes it possible to find a design that is good for everyone.
The distinction you point out between a design error and missing information seems like a helpful distinction to me. Thank you.
It reminds me of the idea of interaction games that CHAI is working on. Instead of having a human give a fully-specified objective to a machine right up front, the idea is to define some protocol for ongoing communication between human and machine about what the objective actually is. When I design practical things I notice this kind of thing happening between me and the world as I start with a vague sense of what my objective is and gradually clarify it through experimentation.
I’m curious to hear about your experience designing and building things, and how it matches up with the model we’re talking about here.
Thanks for the thought-provoking post, Alex.
Thinking about how exactly design stories help create trust, I came upon what might be a useful distinction: whether the design is good according to the considerations known to the designer, vs. whether all relevant considerations are present. A good design story lets us check both of these. The first being false means the designer just did a bad job, or perhaps is hiding something. The second being false means there are actually just considerations the designer didn’t know about—for example because they live implicit in some other human’s head—and spelling things out in a story lets us recognize that, and correct it.
The latter use of stories lets you catch honest mistakes around issues that are unknown unknowns to you, but knowns for someone else. And when I think intuitively about trusting an AI—or another human for that matter—this is a big part of what I care about: beyond them being competent, and not actively deceiving me, I should also trust that they’ll communicate with me enough to fill in all the blind spots they might have about me and the things I care about.
Well said, friend.
Yes when we have a shared understanding of what we’re building together, with honest and concise stories flowing in both directions, we have a better chance of actually understanding what all the stakeholders are trying to achieve, which at least makes it possible to find a design that is good for everyone.
The distinction you point out between a design error and missing information seems like a helpful distinction to me. Thank you.
It reminds me of the idea of interaction games that CHAI is working on. Instead of having a human give a fully-specified objective to a machine right up front, the idea is to define some protocol for ongoing communication between human and machine about what the objective actually is. When I design practical things I notice this kind of thing happening between me and the world as I start with a vague sense of what my objective is and gradually clarify it through experimentation.
I’m curious to hear about your experience designing and building things, and how it matches up with the model we’re talking about here.