Right now, there are ~100 capabilities researchers vs ~30 alignment researchers at OpenAI.
I don’t want to derail this thread, but I do really want to express my disbelief at this number before people keep quoting it. I definitely don’t know 30 people at OpenAI who are working on making AI not kill everyone, and it seems kind of crazy to assert that there are (and I think assertions that there are are the result of some pretty adversarial dynamics I am sad about).
I think a warning shot would dramatically update them towards worry towards worry about accident risk, and therefore I anticipate that OpenAI would drastically shift most of their resources to alignment research. I would guess P(B|A) ~= 80%.
I would like to take bets here, though we are likely to run into doomsday-market problems, though there are ways around that.
I don’t want to derail this thread, but I do really want to express my disbelief at this number before people keep quoting it. I definitely don’t know 30 people at OpenAI who are working on making AI not kill everyone, and it seems kind of crazy to assert that there are (and I think assertions that there are are the result of some pretty adversarial dynamics I am sad about).
I would like to take bets here, though we are likely to run into doomsday-market problems, though there are ways around that.