To me, it just all depends on the estimated percentage that GPT-6 is going to be incrementally “scary smarter” to that person (and what would it mean for that person to translate that into uncontrollability by humans), which are just the conditional probabilities, and the probabilities of the conditions.
Also, in my opinion, we know that many models currently have many problems or known to have expected problems (some are model specific) that are results of misalignment/the way they are optimized, so even if we don’t know what’s going on in the future, addressing these problems are needed anyways, and will be useful overall shared mitigation strategies for any problems coming up.
To me, it just all depends on the estimated percentage that GPT-6 is going to be incrementally “scary smarter” to that person (and what would it mean for that person to translate that into uncontrollability by humans), which are just the conditional probabilities, and the probabilities of the conditions.
Also, in my opinion, we know that many models currently have many problems or known to have expected problems (some are model specific) that are results of misalignment/the way they are optimized, so even if we don’t know what’s going on in the future, addressing these problems are needed anyways, and will be useful overall shared mitigation strategies for any problems coming up.