If we were to blur the name Eliezer Yudkowsky and pretended we saw this a bunch of anonymous people talking on a forum like Reddit, what would your response be to somebody who posted the things Yudkowsky posted above. What pragmatic thing could you tell that person to possibly help them? Every word can be true but It seems overwhelmingly pessimistic in a way that is not helpful, mainly due to nothing in it being actionable.
The position of timelines being too short and the best alignment research being too weak / too slow, while having no avenues or ideas to make things better, with no institution to trust, to the point where we are doomed now, doesn’t lead to a lot of wanting to do anything, which is a guaranteed failure. What should I tell this person? “You seem to be pretty confident in your prediction, what do you suppose you should do to act in accordance with these believes? Wire head yourself and wait for the world to crumble around you? Maybe at least take a break for mental health reasons and let whatever will be will be. If the world is truly doomed you have nothing to worry about anymore”.
I can agree that timelines seem short and good quality alignment research seems rare. I think research like all things humans do follows sturgeon’s law. But the problem is aside from some research with is meant only for prestige building is you can’t tell which will turn out to be crap or not. Nor can you tell where the future will go or what the final outcome will be. We can make use of trends like this person was talking about for predicting the future but there’s always uncertainty. Maybe that’s all this post is which is a rough assessments of one’s personal long term outlook in the field, but it seems pre-mature to say the researchers mentioned in this article are doing things that probably won’t help at this point. With this much pessimism towards our future world we might as well take the low probability of their help working and shoot the moon with it, what have we got to lose in a doomed world?
But that’s the thing, the researchers working on alignment I’m sure will continue doing it even after reading this interaction. If they give up on it we are even more screwed. They might even feel a bit more resentful now knowing what this person thinks about their work, but I don’t think it changed anything.
Maybe I was lucky to get into the AI field within the last couple of years, where short timelines were the expected, rather than something to be feared. I didn’t have the hope of long timelines and now I don’t have to feel crushed by them disappearing (forgive me if I assume too much). We have the time we have to do the best we can, if things take longer, more power too us to get better odds of a good outcome.
Summary: While interesting, this conversation mainly updated me only to the views of the writers, not changing anything pragmatically about my view of research or timelines.
If you think I completely misread the article, and that EY is saying something different than what I interpreted, please let me know.
Every word can be true but It seems overwhelmingly pessimistic in a way that is not helpful, mainly due to nothing in it being actionable.
I’m thinking of this as ‘step one is to figure out our situation; step two is to figure out what to do about it’. Trying too hard to make things actionable from the get-go can interfere with the model-building part.
The position of timelines being too short and the best alignment research being too weak / too slow, while having no avenues or ideas to make things better, with no institution to trust, to the point where we are doomed now, doesn’t lead to a lot of wanting to do anything, which is a guaranteed failure.
Eliezer’s view is ‘the odds of success are low, and I’m pretty uncertain about what paths have the highest EV’, not ‘we’re doomed / the odds of success are negligible’.
If we were to blur the name Eliezer Yudkowsky and pretended we saw this a bunch of anonymous people talking on a forum like Reddit, what would your response be to somebody who posted the things Yudkowsky posted above. What pragmatic thing could you tell that person to possibly help them? Every word can be true but It seems overwhelmingly pessimistic in a way that is not helpful, mainly due to nothing in it being actionable.
The position of timelines being too short and the best alignment research being too weak / too slow, while having no avenues or ideas to make things better, with no institution to trust, to the point where we are doomed now, doesn’t lead to a lot of wanting to do anything, which is a guaranteed failure. What should I tell this person? “You seem to be pretty confident in your prediction, what do you suppose you should do to act in accordance with these believes? Wire head yourself and wait for the world to crumble around you? Maybe at least take a break for mental health reasons and let whatever will be will be. If the world is truly doomed you have nothing to worry about anymore”.
I can agree that timelines seem short and good quality alignment research seems rare. I think research like all things humans do follows sturgeon’s law. But the problem is aside from some research with is meant only for prestige building is you can’t tell which will turn out to be crap or not. Nor can you tell where the future will go or what the final outcome will be. We can make use of trends like this person was talking about for predicting the future but there’s always uncertainty. Maybe that’s all this post is which is a rough assessments of one’s personal long term outlook in the field, but it seems pre-mature to say the researchers mentioned in this article are doing things that probably won’t help at this point. With this much pessimism towards our future world we might as well take the low probability of their help working and shoot the moon with it, what have we got to lose in a doomed world?
But that’s the thing, the researchers working on alignment I’m sure will continue doing it even after reading this interaction. If they give up on it we are even more screwed. They might even feel a bit more resentful now knowing what this person thinks about their work, but I don’t think it changed anything.
Maybe I was lucky to get into the AI field within the last couple of years, where short timelines were the expected, rather than something to be feared. I didn’t have the hope of long timelines and now I don’t have to feel crushed by them disappearing (forgive me if I assume too much). We have the time we have to do the best we can, if things take longer, more power too us to get better odds of a good outcome.
Summary: While interesting, this conversation mainly updated me only to the views of the writers, not changing anything pragmatically about my view of research or timelines.
If you think I completely misread the article, and that EY is saying something different than what I interpreted, please let me know.
I’m thinking of this as ‘step one is to figure out our situation; step two is to figure out what to do about it’. Trying too hard to make things actionable from the get-go can interfere with the model-building part.
Eliezer’s view is ‘the odds of success are low, and I’m pretty uncertain about what paths have the highest EV’, not ‘we’re doomed / the odds of success are negligible’.
Continuously describing a situation accurately is a key action item.