The paper starts with the assumption that humans will create many AI-agents and assign some of them selfish goals and that combined with competitive pressure and other factors may presumably create a Molochy -situation where most selfish and immoral AI’s will propagate and evolve—leading to loss of control and downfall of the human race. The paper in fact does not advocate the idea of a single AI foom. While the paper itself makes some valid points it does not answer my initial question and critique of OP.
The question as stated was: But WHY would the AGI “want” anything at all unless humans gave it a goal(/s)?
ok how’s this then https://arxiv.org/abs/2303.16200
The paper starts with the assumption that humans will create many AI-agents and assign some of them selfish goals and that combined with competitive pressure and other factors may presumably create a Molochy -situation where most selfish and immoral AI’s will propagate and evolve—leading to loss of control and downfall of the human race. The paper in fact does not advocate the idea of a single AI foom. While the paper itself makes some valid points it does not answer my initial question and critique of OP.
Fair enough.