Note that Richard is not treating knightian uncertainty as special and unquantifiable, but instead is giving examples of how to treat it like any other uncertainty, that he is explicitly quantifying and incorporating in his predictions.
I’d prefer calling Richard’s “model error” to separate the two, but I’m also okay appropriating the term as Richard did to point to something coherent.
To my knowledge, we currently don’t have a way of translating statements about “loss” into statements about “real-world capabilities”.
My intuition is that it’s not a great approximation in those cases, similar to how in regular Laplace the empirical approximation is not great when you have eg N<5
Id need to run some calculations to confirm that intuition though.
Here is a 2012 meme about SolidGoldMagikarp
This site claims that the strong SolidGoldMagikarp was the username of a moderator involved somehow with Twitch Plays Pokémon
https://infosec.exchange/@0xabad1dea/109813506433583177
I still don’t understand—did you mean “when T/t is close to zero”?
What’s r?
That’s exactly right, and I think the approximation holds as long as T/t>>1.
This is quite intuitive—as the amount of data goes to infinity, the rate of events should equal the number of events so far divided by the time passed.
If you want to join the Spanish-speaking EA community, you can do so through this link!
I agree with the sentiment that indiscriminate regulation is unlikely to have good effects.
I think the step that is missing is analysing the specific policies No-AI Art Activist are likely to advocate for, and whether it is a good idea to support it.
My current sense is that data helpful for alignment is unlikely to be public right now, and so harder copyright would not impede alignment efforts. The kind of data that I could see being useful are things like scores and direct feedback. Maybe at most things like Amazon reviews could end up being useful for toy settings.
Another aspect that the article does not touch on is that copyright enforcement could have an adverse effect. Currently there is basically no one trying to commercialize training dataset curation because enforcing copyright use is a nightmare. It is in fact a common good. I’d expect there would be more incentives to create large curated datasets if this was not the case.
Lastly, here are some examples of “no AI art” legislation I expect the movement is likely to support:
Removing copyright protection of AI generated images
Enforcing AI training data to be strictly opt-in
Forcing AI content to be labelled as such
Besides regulation, I also expect activists to 4) pressure companies to deboost AI made content in social medial sites.
My general impression is that 3) is slightly good for AI safety. People in the AI Safety community have advocated for it in the past, convincingly.
I’m more agnostic on 1), 2) and 4).
1 and 4 will make AI generation less profitable, but also it’s somewhat confused—it’s a weird double standard to apply to AI content over human made content.
2 makes training more annoying, but could lead to commercialization of datasets and more collective effort being put into building them. I also think there is a possibly a coherent moral case for it, which I’m still trying to make my mind about, regardless of the AI safety consequences.
All in all, I am confused, though I wholeheartedly agree that we should be analysing and deciding to support specific policies rather than eg the anti AI art movement as a whole.
Great work!
Stuart Armstrong gave one more example of a heuristic argument based in the presumption of independence here.
Here are my quick takes from skimming the post.
In short, the arguments I think are best are A1, B4, C3, C4, C5, C8, C9 and D. I don’t find any of them devastating.
A1. Different calls to ‘goal-directedness’ don’t necessarily mean the same concept
I am not sure I parse this one.I am reading it as “AI systems might be more like imitators than optimizers” from the example, which I find moderately persuasive
A2. Ambiguously strong forces for goal-directedness need to meet an ambiguously high bar to cause a risk
I am not sure I understand this one either.I am reading it as “there might be no incentive for generality” which I dont find persuasive—I think there is a strong incentive
B1. Small differences in utility functions may not be catastrophic
I dont find this persuasive. I think the evidence from optimization theory setting variables to extreme values is suggestive enough to suggest this is not the default
B2. Differences between AI and human values may be small
B3. Maybe value isn’t fragile
The only example we have of general intelligence (humans) seems to have strayed pretty far from evolutionary incentives, so I find this unpersuasive
B4. [AI might only care about]Short-term goals
I find that somewhat persuasive, or at least not obviously wrong, similar to A1. There is a huge incentive for instilling long term thinking though.
C1. Human success isn’t from individual intelligence
I dont find this persuasive. Im not convinced there is a meaningful difference between “a single AGI” and “a society of AGIs”. A single AGI could be running a billion independent threads of thought and outspeed humans.
C2. AI agents may not be radically superior to combinations of humans and non-agentic machines
I dont find this persuasive. Seems unlikely that human-in-the-loop is going to have any advantages over pure machines.
C3. Trust
I find this plausible but not convincing
C4. Headroom
Plausible but not convincing. I dont find any of the particular examples of lack of headroom convincing, and I think the prior should be that there is a lot of headroom
C5. Intelligence may not be an overwhelming advantage
I find this moderately persuasive though not entirely convincing
C6. Unclear that many goals realistically incentivise taking over the universe
I find this unconvincing. I think there are many reasons to expect that taking over the universe is a convergent goal.
C7. Quantity of new cognitive labor is an empirical question, not addressed
I dont find this superpersuasive. In particular I think there is a good chance that once we have AGI we will be in a hardware overhang and be able to execute tons of AGI-equivalents
C8. Speed of intelligence growth is ambiguous
I find this plausible
C9. Key concepts are vague
Granted but not a refutation in itself
D1. The argument overall proves too much about corporations
I find this somewhat persuasive
Eight examples, no cherry-picking:
Nit: Having a wall of images makes this post unnecessarily harder to read.
I’d recommend making a 4x2 collage with the photos so they don’t take that much space.
Because there is more data available for FP32, so it’s easier to study trends there.
We should release a piece soon about how the picture changes when you account for different number formats, plus considering that most runs happen with hardware that is not the most cost-efficient.