That’s right and that’s a consequence of uncertainty, which prevents us from bounding risks. Decreasing uncertainty (e.g. through modelling or through the ability to set bounds) is the objective of risk management.
Doses of radiation are quite predictable
I think it’s mostly in hindsight. When you read stuff about nuclear safety in the 1970s, it’s really not how it was looking.
See Section 2
the arc of new technology is not [predictable]
I think that this sets a “technology is magic” vibe which is only valid for scaling neural nets (and probably only because we haven’t invested that much into understanding scaling laws etc.), and not for most other technologies. We can actually develop technology where we know what it’s doing before building it and that’s what we should aim for given what’s at stakes here.
Thanks for your comment.
That’s right and that’s a consequence of uncertainty, which prevents us from bounding risks. Decreasing uncertainty (e.g. through modelling or through the ability to set bounds) is the objective of risk management.
I think it’s mostly in hindsight. When you read stuff about nuclear safety in the 1970s, it’s really not how it was looking.
See Section 2
I think that this sets a “technology is magic” vibe which is only valid for scaling neural nets (and probably only because we haven’t invested that much into understanding scaling laws etc.), and not for most other technologies. We can actually develop technology where we know what it’s doing before building it and that’s what we should aim for given what’s at stakes here.