Moloch is not, in my opinion, the correct analogy to use for systemic problems.
Moloch absolves humans of guilt for systemic problems by hand waving them away as self interest.
But systemic problems usually stem from things that are both bad when done in aggregate and bad for us as individuals. Hence why my analogy used a button that would give one nothing of value, just something we from our blind position would think to be of value.
I agree with the rest of your comment but it’s trivial to replicate the work by anthropic or openai in a focused direction. It’s already been done.
Their main edge is the RLHF datasets they have but those are good for safety (arguably, ad you point out) and for capabilities in-so-far as they are interacting with humans that haven’t trained to use such systems.
So we do and likely will live in the hardest of worlds, where it’s all open source.
Moloch is not, in my opinion, the correct analogy to use for systemic problems.
Moloch absolves humans of guilt for systemic problems by hand waving them away as self interest.
But systemic problems usually stem from things that are both bad when done in aggregate and bad for us as individuals. Hence why my analogy used a button that would give one nothing of value, just something we from our blind position would think to be of value.
I agree with the rest of your comment but it’s trivial to replicate the work by anthropic or openai in a focused direction. It’s already been done.
Their main edge is the RLHF datasets they have but those are good for safety (arguably, ad you point out) and for capabilities in-so-far as they are interacting with humans that haven’t trained to use such systems.
So we do and likely will live in the hardest of worlds, where it’s all open source.