This reminds me of a passage in Richard Feynman’s memoir “What do you care what other people think?”. Four pages into the chapter Gumshoes, (page 163 in the Unwin Paperback edition):
Then this business of Thiokol changing its position came up. Mr. Rogers and Dr. Ride were asking two Thiokol managers, Mr. Mason and Mr. Lund, how many people were against the launch, even at the last moment.
“We didn’t poll everyone,” says Mr. Mason.
“Was there a substantial number against the launch, or just one or two?”
“There were, I would say, probably five or six in engineering who at that point would have said it is not as conservative to go with that temperature, and we don’t know. The issue was we didn’t know for sure that it would work.”
“So it was evenly divided?”
“That’s a very estimated number.”
It struck me that the Thiokol managers were waffling. But I only knew how to ask simpleminded questions. So I said, “Could you tell me, sirs, the names of your four best seals experts, in order of ability?”
“Roger Boisjoly and Arnie Thompson are one and two. Then there’s Jack Kapp, and, uh … Jerry Burns.”
I turned to Mr. Boisjoly, who was right there, at the meeting. “Mr. Boisjoly, were you in agreement that it was okay to fly?”
He says, “No, I was not.”
I ask Mr. Thompson, who was also there.
“No. I was not.”
I say “Mr. Kapp?”
Mr. Lund says, “He is not here, I talked to him after the meeting, and he said, ‘I would have made that decision, given the information we had.’”
“And the fourth man?”
“Jerry Burns. I don’t know what his position was.”
“So,” I said, “of the four, we have one ‘don’t know,’ one ‘very likely yes,’ and the two who were mentioned right away as being the best seal experts, both said no.” So this “evenly split” stuff was a lot of crap. The guys who knew the most about the seals—what were they saying?
That is the end of that section of the chapter and Feynman turns to the infra-red thermometer and the temperatures on the launch pad.
That was my introduction to this aspect of bureaucratic infighting. The bureaucrat asks his technical experts, the one closest to the issue. If he gets the answer that he wants, it is accepted. If not, he widens the pool of experts. Those too close to the issue are at risk of ignoring the social cues to the desired answer, but the wider pool of experts can be more flexible at responding to the broader social context. Then the bureaucrat gets to take an unweighted average (that is not weighting the original experts more highly). Which boosts the probability of getting the desired answer and reduces the probability of getting the correct answer.
Back in 1988 this was perhaps a busted technique. But that was many years ago. The notion of broadening your survey of experts seems to be back in fashion.
This reminds me of a passage in Richard Feynman’s memoir “What do you care what other people think?”. Four pages into the chapter Gumshoes, (page 163 in the Unwin Paperback edition):
Then this business of Thiokol changing its position came up. Mr. Rogers and Dr. Ride were asking two Thiokol managers, Mr. Mason and Mr. Lund, how many people were against the launch, even at the last moment.
“We didn’t poll everyone,” says Mr. Mason.
“Was there a substantial number against the launch, or just one or two?”
“There were, I would say, probably five or six in engineering who at that point would have said it is not as conservative to go with that temperature, and we don’t know. The issue was we didn’t know for sure that it would work.”
“So it was evenly divided?”
“That’s a very estimated number.”
It struck me that the Thiokol managers were waffling. But I only knew how to ask simpleminded questions. So I said, “Could you tell me, sirs, the names of your four best seals experts, in order of ability?”
“Roger Boisjoly and Arnie Thompson are one and two. Then there’s Jack Kapp, and, uh … Jerry Burns.”
I turned to Mr. Boisjoly, who was right there, at the meeting. “Mr. Boisjoly, were you in agreement that it was okay to fly?”
He says, “No, I was not.”
I ask Mr. Thompson, who was also there.
“No. I was not.”
I say “Mr. Kapp?”
Mr. Lund says, “He is not here, I talked to him after the meeting, and he said, ‘I would have made that decision, given the information we had.’”
“And the fourth man?”
“Jerry Burns. I don’t know what his position was.”
“So,” I said, “of the four, we have one ‘don’t know,’ one ‘very likely yes,’ and the two who were mentioned right away as being the best seal experts, both said no.” So this “evenly split” stuff was a lot of crap. The guys who knew the most about the seals—what were they saying?
That is the end of that section of the chapter and Feynman turns to the infra-red thermometer and the temperatures on the launch pad.
That was my introduction to this aspect of bureaucratic infighting. The bureaucrat asks his technical experts, the one closest to the issue. If he gets the answer that he wants, it is accepted. If not, he widens the pool of experts. Those too close to the issue are at risk of ignoring the social cues to the desired answer, but the wider pool of experts can be more flexible at responding to the broader social context. Then the bureaucrat gets to take an unweighted average (that is not weighting the original experts more highly). Which boosts the probability of getting the desired answer and reduces the probability of getting the correct answer.
Back in 1988 this was perhaps a busted technique. But that was many years ago. The notion of broadening your survey of experts seems to be back in fashion.