Like, during the time that MIRI was non-non-disclosed (i.e. open), I don’t think there was substantially more checking of their work than there is now. Most of academia and industry has/had no interest. There were LWers like Paul Christiano and Wei Dai and Stuart Armstrong that engaged, and I’m sad that these people aren’t able to engage with the present work. But at the time Paul and Wei were basically saying “I don’t get why this is going to work out” so it’s not like MIRI could start getting much more negative feedback, unless Paul and Wei were going to say “This obviously fails and I can prove how,” which I assign little probability to.
However, the open Philanthropy project did have people evaluate Miri’s work, and they ended up sending them a substantial donation because they liked the Logical Induction paper – unless I’m misremembering how it went down.
Yes, that’s right. OTOH I recall OpenPhil trying to evaluate MIRI’s work before the logical induction paper, and they thought it was pretty terrible. As I mentioned, I’d be pro MIRI writing up any / a few clear results over the next few years, for reasons like this.
There’s also a question of how much OpenPhil’s support mattered in your estimation of MIRI. I might write more on it later, but overall it’s not been a major factor for me.
However, the open Philanthropy project did have people evaluate Miri’s work, and they ended up sending them a substantial donation because they liked the Logical Induction paper – unless I’m misremembering how it went down.
Yes, that’s right. OTOH I recall OpenPhil trying to evaluate MIRI’s work before the logical induction paper, and they thought it was pretty terrible. As I mentioned, I’d be pro MIRI writing up any / a few clear results over the next few years, for reasons like this.
There’s also a question of how much OpenPhil’s support mattered in your estimation of MIRI. I might write more on it later, but overall it’s not been a major factor for me.
See also My current thoughts on MIRI’s “highly reliable agent design” work by Daniel Dewey (Open Phil lead on technical AI grant-making).
From the “What do I think of HRAD?” section: