I don’t think any one option is precise enough that it’s correct on its own, so I will have to say “5” as well.
Here’s my take:
Altruism can be a result of both good and bad mental states.
Helping others tends to be good for them, at least temporarily.
Helping people can prevent them from helping themselves, and from growing.
Helping something exist which wouldn’t exist without your help is to get in the way of natural selection, which over time can result in many groups who are a net negative for society in that they require more than they provide. They might also remain dependent on others.
Finally, (and I expect some people to disagree with this) I think that moral good is a luxury. Luxuries are pleasant, but expensive, so when you engage in more luxury than you can afford, it stops being sustainable. And putting luxuries above necessities seem to me a good definition of decadence.
Everything has dose-dependent and context-dependent pros and cons.
I think you’re expecting too much of the word “good”. I don’t think any “good” exists such that more of it is always better, so I think “good” is a region of space rather than a direction. If optimization is gradient descent, then the “good” direction might change with every step you take. But if optimization means “what metric should we optimize for?” then we don’t know (we have yet to find a single metric which an AGI could maximize without destroying humanity. Heading too far in any direction seems dangerous). So I think many peoples intuition of the word “good” can prevent them from ever hitting a satisfactory answer (as they’re actually searching for something which can be taken to infinity without anything bad happening as a result, and not even considering the context in question)
I don’t think any one option is precise enough that it’s correct on its own, so I will have to say “5” as well.
Here’s my take:
Altruism can be a result of both good and bad mental states.
Helping others tends to be good for them, at least temporarily.
Helping people can prevent them from helping themselves, and from growing.
Helping something exist which wouldn’t exist without your help is to get in the way of natural selection, which over time can result in many groups who are a net negative for society in that they require more than they provide. They might also remain dependent on others.
Finally, (and I expect some people to disagree with this) I think that moral good is a luxury. Luxuries are pleasant, but expensive, so when you engage in more luxury than you can afford, it stops being sustainable. And putting luxuries above necessities seem to me a good definition of decadence.
Everything has dose-dependent and context-dependent pros and cons.
I think you’re expecting too much of the word “good”. I don’t think any “good” exists such that more of it is always better, so I think “good” is a region of space rather than a direction. If optimization is gradient descent, then the “good” direction might change with every step you take. But if optimization means “what metric should we optimize for?” then we don’t know (we have yet to find a single metric which an AGI could maximize without destroying humanity. Heading too far in any direction seems dangerous). So I think many peoples intuition of the word “good” can prevent them from ever hitting a satisfactory answer (as they’re actually searching for something which can be taken to infinity without anything bad happening as a result, and not even considering the context in question)