the most efficient way to train Transformer-based language models is to train very large models and stop before convergence, rather than training smaller models to convergence.
I understand this refers to some notion of economic efficiency, but what is meant by convergence? Getting a perfect score on the training set?
Commentary
I think that a large body of historical literature supports the conclusion that American civilizations fell primarily because of their exposure to diseases which they lacked immunity to, rather than because of European military power.
European biological military power
I think the primary insight here should instead be that pandemics can kill large groups of humans, and therefore it would be worth exploring the possibility that AI systems use pandemics as a mechanism to kill large numbers of biological humans.
Non-pandemic conditions as infrastructure:
Humans which rely on other humans in ways that involve contact with other humans, directly or indirectly*, are subject to, and are themselves, a vector for disease/viruses.
*Such as via shared surfaces, like door nobs.
Though we do not usually think of “there not being a pandemic” as infrastructure, systems which operate under the assumption that there isn’t one are necessarily vulnerable to such. Methods, procedures, and work that can make this state of affairs more stable/robust may serve a similar role to infrastructure designed with the possibility of earthquakes in mind—perhaps useless when things are still, but an incredibly important foundation when a quake hits.
Rohin’s opinion: It makes sense that role playing can help find extreme, edge case scenarios.
I’d say it might be useful to find obvious scenarios—though if the tech tree is an abstract thing rather than one involving making decisions it might not enable that.
Made up examples:
Before we release our new image recognition system to the public, let’s quietly test it on a database of celebrities’ facebook profile pictures and check the results. (And which celebrity does it think I look most like?)
There have been some concerns about how malicious actors could use this tech. So as CEO I suggest we see if this can be used to impersonate me.
Instead of pursuing just one approach, let’s try multiple in parallel, possibly combining different approaches to see if that improves performance...
I may not know what a “general quantum computer” is, but if it can’t make our search algorithms better or beat humans at video games, what did we get it for?
Some of the difficult about this being realistic is expertise/accessibility, though it might be interesting to see how far people can get with black box outlines or extrapolation.
I understand this refers to some notion of economic efficiency, but what is meant by convergence? Getting a perfect score on the training set?
Convergence = Further training doesn’t improve training / validation accuracy.
style commentary:
Thanks. The current publishing system doesn’t like bullet points very much, but I probably should have done the hacky version where I just put—on new lines.
Questions
I understand this refers to some notion of economic efficiency, but what is meant by convergence? Getting a perfect score on the training set?
Commentary
European biological military power
Non-pandemic conditions as infrastructure:
Humans which rely on other humans in ways that involve contact with other humans, directly or indirectly*, are subject to, and are themselves, a vector for disease/viruses.
*Such as via shared surfaces, like door nobs.
Though we do not usually think of “there not being a pandemic” as infrastructure, systems which operate under the assumption that there isn’t one are necessarily vulnerable to such. Methods, procedures, and work that can make this state of affairs more stable/robust may serve a similar role to infrastructure designed with the possibility of earthquakes in mind—perhaps useless when things are still, but an incredibly important foundation when a quake hits.
I’d say it might be useful to find obvious scenarios—though if the tech tree is an abstract thing rather than one involving making decisions it might not enable that.
Made up examples:
Before we release our new image recognition system to the public, let’s quietly test it on a database of celebrities’ facebook profile pictures and check the results. (And which celebrity does it think I look most like?)
There have been some concerns about how malicious actors could use this tech. So as CEO I suggest we see if this can be used to impersonate me.
Instead of pursuing just one approach, let’s try multiple in parallel, possibly combining different approaches to see if that improves performance...
I may not know what a “general quantum computer” is, but if it can’t make our search algorithms better or beat humans at video games, what did we get it for?
Some of the difficult about this being realistic is expertise/accessibility, though it might be interesting to see how far people can get with black box outlines or extrapolation.
style commentary:
“Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society (Carina Prunkl and Jess Whittlestone) (summarized by Rohin): This paper argues that the existing near-term / long-term distinction conflates four different axes on which research could differ: ”
the capability level of AI systems (current pattern-matching systems vs. future intelligent systems)
the impacts of AI systems (impacts that are being felt now like fairness vs. ones that will be felt in the future like x-risks)
certainty (things that will definitely be problems vs. risks that are more speculative)
extremity (whether to prioritize particularly extreme risks)
Convergence = Further training doesn’t improve training / validation accuracy.
Thanks. The current publishing system doesn’t like bullet points very much, but I probably should have done the hacky version where I just put—on new lines.