I wonder if the Facebook algorithm is a good example of the counterintuitive difficulty of alignment (as a more general concept).
You’re trying to figure out the best posts and comments to prioritize in the feed. So you look at things like upvotes, page views and comment replies. But it turns out that that captures things like how much of a demon thread it is. Who would have thought metrics like upvotes and page views could be so… demonic?
I don’t think this is an alignment-is-hard-because-it’s-mysterious, I think it’s “FB has different goals than me”. FB wants engagement, not enjoyment. I am not aligned with FB, but FB’s algorithm is pretty aligned with its interests.
Oh yeah, that’s a good point. I was thinking about Facebook actually having the goal to promote quality content. I think I remember hearing something about how that was their goal at first, then they got demon stuff, then they realized demon stuff made them the most money and kept doing it. But still, people don’t associate Facebook with having the goal of promoting quality content, so I don’t think it’s a good example of the counterintuitive difficulty of alignment.
I wonder if the Facebook algorithm is a good example of the counterintuitive difficulty of alignment (as a more general concept).
You’re trying to figure out the best posts and comments to prioritize in the feed. So you look at things like upvotes, page views and comment replies. But it turns out that that captures things like how much of a demon thread it is. Who would have thought metrics like upvotes and page views could be so… demonic?
I don’t think this is an alignment-is-hard-because-it’s-mysterious, I think it’s “FB has different goals than me”. FB wants engagement, not enjoyment. I am not aligned with FB, but FB’s algorithm is pretty aligned with its interests.
Oh yeah, that’s a good point. I was thinking about Facebook actually having the goal to promote quality content. I think I remember hearing something about how that was their goal at first, then they got demon stuff, then they realized demon stuff made them the most money and kept doing it. But still, people don’t associate Facebook with having the goal of promoting quality content, so I don’t think it’s a good example of the counterintuitive difficulty of alignment.