Shouldn’t the very first goal be to fully define an ethical theory of friendliness before even starting on a goal system to implement the theory? I have some doubts that an acceptable theory can be formalized. Our ethical systems are so human-centric that formalizing them in rational terms will likely lead to either a very weak human-centric theory with potential loopholes or a general theory of ethics that places no particular importance on the concerns of humans, even large groups of them. For instance, I find it much more likely that a general theory of ethics would have little ability to differentiate between the desires of rocks and dirt, bacteria, and humans. What measurable properties of humans makes our actions worthy of more attention than the actions of bacteria or of a solar system or galaxy? Is it the ability for self-referential thought, a consciousness, or conscience? Even assuming a suitable definition for a class of beings whose utilities are valued can be found, the weighting of utility for each being will almost certainly not be in our favor. There are potentially infinitely many more potential future beings whose values and existence depend on actions in the present. There are potentially existing extraterrestrial beings whose values must be considered in case we ever affect them.
Maybe I have missed important discussions where the details of friendliness theory have already been hashed out. If so, I apologize and would appreciate links to them.
Shouldn’t the very first goal be to fully define an ethical theory of friendliness before even starting on a goal system to implement the theory? I have some doubts that an acceptable theory can be formalized. Our ethical systems are so human-centric that formalizing them in rational terms will likely lead to either a very weak human-centric theory with potential loopholes or a general theory of ethics that places no particular importance on the concerns of humans, even large groups of them. For instance, I find it much more likely that a general theory of ethics would have little ability to differentiate between the desires of rocks and dirt, bacteria, and humans. What measurable properties of humans makes our actions worthy of more attention than the actions of bacteria or of a solar system or galaxy? Is it the ability for self-referential thought, a consciousness, or conscience? Even assuming a suitable definition for a class of beings whose utilities are valued can be found, the weighting of utility for each being will almost certainly not be in our favor. There are potentially infinitely many more potential future beings whose values and existence depend on actions in the present. There are potentially existing extraterrestrial beings whose values must be considered in case we ever affect them.
Maybe I have missed important discussions where the details of friendliness theory have already been hashed out. If so, I apologize and would appreciate links to them.