Yes, agreed, a researcher saying “give me $5 to research technology with implausible payoff” is just some guy saying “give me $5 to use my implausible powers” with different paint and has many of the same problems.
The scenario I’m thinking of is “I have, after doing a bunch of research, discovered some novel physics which, given my understanding of it and the experimental data I’ve gathered, implies the ability to improve (really big number) lives,” which raises the possibility that I ought to reject the results of my own experiments and my own theorizing, because the conclusion is just so bloody implausible (at least when expressed in human terms; EY loses me when he starts talking about quantifying the implausibility of the conclusion in terms of bits of evidence and/or bits of sensory input and/or bits of cognitive state).
And in particular, the “you could just as easily harm (really big number) lives!” objection simply disappears in this case; it’s no more likely than anything else, and vanishes into unconsiderability when compared to “nothing terribly interesting will happen,” unless I posit that I actually do know what I’m doing.
Yes, agreed, a researcher saying “give me $5 to research technology with implausible payoff” is just some guy saying “give me $5 to use my implausible powers” with different paint and has many of the same problems.
The scenario I’m thinking of is “I have, after doing a bunch of research, discovered some novel physics which, given my understanding of it and the experimental data I’ve gathered, implies the ability to improve (really big number) lives,” which raises the possibility that I ought to reject the results of my own experiments and my own theorizing, because the conclusion is just so bloody implausible (at least when expressed in human terms; EY loses me when he starts talking about quantifying the implausibility of the conclusion in terms of bits of evidence and/or bits of sensory input and/or bits of cognitive state).
And in particular, the “you could just as easily harm (really big number) lives!” objection simply disappears in this case; it’s no more likely than anything else, and vanishes into unconsiderability when compared to “nothing terribly interesting will happen,” unless I posit that I actually do know what I’m doing.