I’m a researcher on the technical governance team at MIRI.
Views expressed are my own, and should not be taken to represent official MIRI positions. Similarly, views within the technical governance team do vary.
Previously:
Helped with MATS, running the technical side of the London extension (pre-LISA).
Worked for a while on Debate (this kind of thing).
Quick takes on the above:
I think MATS is great-for-what-it-is. My misgivings relate to high-level direction.
Worth noting that PIBBSS exists, and is philosophically closer to my ideal.
The technical AISF course doesn’t have the emphasis I’d choose (which would be closer to Key Phenomena in AI Risk). It’s a decent survey of current activity, but only implicitly gets at fundamentals—mostly through a [notice what current approaches miss, and will continue to miss] mechanism.
I don’t expect research on Debate, or scalable oversight more generally, to help significantly in reducing AI x-risk. (I may be wrong! - some elaboration in this comment thread)
I don’t think this is quite right.
Two major objections to the bio-anchors 30-year-median conclusion might be:
The whole thing is laundering vibes into credible-sounding headline numbers.
Even if we stipulate that the methodology is sound, it measures an upper bound, not a median.
To me, (2) is the more obvious error. I basically buy (1) too, but I don’t think we’ve gotten empirical evidence, since (2).
I guess there’s a sense in which a mistake on (2) could be seen as a consequence of (1) - but it seems distinct: it’s a logic error, not a free parameter. I do think it’s useful to distinguish [motivated reasoning in free-parameter choice] from [motivated reasoning in error-checking].
It’s not so obvious to me that the bio-anchors report was without foundation as an upper bound estimate.