I believe those are useful frames for understanding the impacts.
Chris_Leong
Karma: 7,545
To “misuse” to me implies taking a bad action. Can you explain what misuse occurred here?
They’re recklessly accelerating AI. Or, at least, that’s how I see it. I’ll leave it to others to debate whether or not this characterisation is accurate.Obviously details matter
Details matter. It depends on how bad it is and how rare these actions are.
This seems like a better solution on the surface, but once you dig in, I’m not so sure.
Once you hire someone, assuming they’re competent, it’s very hard for you to decide to permanently bar them from gaining a leadership role. How are you going to explain promoting someone who seems less competent than them to a leadership role ahead of them? Or is the plan to never promote them and refuse to ever discuss it, which would create weird dynamics within an organisation.
I would love to hear if you think otherwise, but it seems unworkable to me.
Everything has trade-offs.
I agree that attempting to be 100% sure that they’re responsible would be a mistake. Specifically, the unwanted impacts would likely be too high.
I agree that this issue is complex and I don’t pretend to have all of the solutions.
I just think it’s really bad if people feel that they can’t speak relatively freely with the forecasting organisations because they’ll misuse the information. I think this is somewhat similar to how it is important for folks to be able to speak freely to their doctor/lawyer/psychologist though I admit that the analogy isn’t perfect and that straightforwardly copying these norms over would probably be a mistake.Nonetheless, I think it is worthwhile discussing whether there should be some kind of norms and what they should be. As you’ve rightly pointed out, are a lot of issues that would need to be considered. I’m not saying I know exactly what these norms should be. I see myself as more just starting a discussion.
(This is distinct from my separate point about it being a mistake to hire folk who do things like this. It is a mistake to have hired folks who act strongly against your interests even if they don’t break any ethical injuctions)
Thanks for sharing your thoughts.
I agree that humans with wise AI advisors is a more promising approach, at least at first, then attempting to directly program wisdom into an autonomously acting agent.
Beyond that, I personally haven’t made up my mind yet about the best way to use wisdom tech.
I would like to see serious thought given to instituting such a norm. There’s a lot of complexities here, figuring out what is or isn’t kosher would be challenging, but it should be explored.
Agreed. This is how these codes form. Someone does something like this and then people discuss and decide that there should be a rule against it or that it should at least be frowned upon.
Thanks for weighing in.
This attitude feels like a recipe for creating an intellectual bubble
Oh, additional screening could very easily have unwanted side-effects. That’s why I wrote: “It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias” and why it would be better for this issue to never have arisen in the first place. Actions like this can create situations with no good trade-offs.
I think it would be pretty bad for the AI safety community if it just relied on forecasting work from card-carrying AI safety advocates.
I was definitely not suggesting that the AI safety community should decide which forecasts to listen to based on the views of the forecasters. That’s irrelevant, we should pay attention to the best forecasters.
I was talking about funding decisions. This is a separate matter.If someone else decides to fund a forecaster even though we’re worried they’re net-negative or they do work voluntarily, then we should pay attention to their forecasts if they’re good at their job.
Of course people will use the knowledge they gain in collaboration with you for the purposes that they think are best
Seems like several professions have formal or informal restrictions on how they can use information that they gain in a particular capacity to their advantage. People applying for a forecasting role are certainly entitled to say, ’If I learn anything about AI capabilities here, I may use it to start an AI startup and I won’t actually feel bad about this”. It doesn’t mean you have to hire them.
This post is now looking extremely prescient.
I guess orgs need to be more careful about who they hire as forecasting/evals researchers.
Sometimes things happen, but three people at the same org...
This is also a massive burning of the commons. It is valuable for forecasting/evals orgs to be able to hire people with a diversity of viewpoints in order to counter bias. It is valuable for folks to be able to share information freely with folks at such orgs without having to worry about them going off and doing something like this.But this only works if those less worried about AI risks who join such a collaboration don’t use the knowledge they gain to cash in on the AI boom in an acceleratory way. Doing so undermines the very point of such a project, namely, to try to make AI go well. Doing so is incredibly damaging to trust within the community.
Now let’s suppose you’re an x-risk funder considering whether to fund their previous org. This org does really high-quality work, but the argument for them being net-positive is now significantly weaker. This is quite likely to make finding future funding harder for them.This is less about attacking those three folks and more just noting that we need to strive to avoid situations where things like this happen in the first place. This requires us to be more careful in terms of who gets hired.
The corollary of this is that an aligned ASI in the strong sense of “aligned” used here would have to dissolve currently existing human institutions, and the latter will obviously oppose that
Interesting analysis, but this statement is a bit strong. A global safe AI project would be theoretically possible, but would be extremely challenging to solve the co-ordination issues without AI progress dramatically slowing. Then again, all plans are challenging/potentially impossible.Alternatively, an aligned ASI could be explicitly instructed to preserve existing institutions. Perhaps it’d be limited to providing advice, or (stronger) it wouldn’t intervene except by preventing existential or near-existential risks.
Yet another possibility is that the world splits into factions which produce their own AGI’s and then these AGIs merge.
A fourth option would be to negotiate a deal where only a few countries are allowed to develop AGI, but in exchange, the UN gets to send observers and provide input on the development of the technology.
This article is extremely well written and I really appreciated how well he supported his positions with facts.
However, this article seems to suggest that he doesn’t quite understand the argument for making alignment the priority. This is understandable as it’s rarely articulated clearly. The core limitation of differential tech development/d/acc/coceleration is that these kinds of imperfect defenses only buy time (this judgment can be justified with the articles he provides in his article). An aligned ASI, if it were possible, would be capable of a degree of perfection beyond that of human institutions. This would give us a stable long-term solution. Plans that involve less powerful AIs or a more limited degree of alignment mostly do not.
Great post.
One thing that help clarify the cup example: if I look at the cup without trying to interpret it, then I can often find myself seeing separate images coming from each eye.
It’s possible to doubt this, maybe I’m unconsciously shifting the direction my eyes point and the experience before was only a single cup, but regular science experiments can provide incorrect results too.
What empirical research directions has Eliezer commented positively on?
Maybe you’re thinking younger than I was thinking.
I expect you’d mostly want folks who’d already completed an undergraduate degree, with sufficiently talented folks being pulled in earlier.
Funnily enough I was thinking about this yesterday and wondering if I’d be able to find it, so great timing! Thanks for the comment.
Yeah, I definitely have worries about this as well. Nonetheless, I would prefer for discussion to happen somewhere rather than nowhere at all.
I might comment there, but it’s hard to know how busy I am.
How long does SPARC go for?
I used to really like Matthew Barnett’s posts as providing contrarian but interesting takes.
However, over the last few years, I’ve started to few more negatively about them. I guess I feel that his posts tend to be framed in a really strange way such that, even though there’s often some really good research there, it’s more likely to confuse the average reader than anything else and even if you can untangle the frames, I usually don’t find worth it the time.
I should mention though that as I’ve started to feel more negative about them, I’ve started to read less of them and to engage less deeply with the ones I do look it, so there’s a chance my view would be different if I read more.
I’d probably feel more positive about any posts he writes that are closer to presenting data and further away from interpretation.
That said, Epoch overall has produced some really high-quality content and I’d definitely like to see more independent scenes.