good capabilities form something like an attractor well
In my own experience examining the foundations of things in the world, I have repeatedly found there to be less of an attractor-of-fundamentally-effective-decision-making than I had anticipated. In every way that I expected to find such an attractor within epistemology, mathematics, empiricism, ethics, I found in fact that even the very basic assumptions that I started with were unfounded, and found nothing firm to replace with them with. Probability theory: not a fundamental answer to epistemology; proof-based agents: deeply paradoxical and seemingly untrustworthy; the scientific method: not precise enough to be a final answer to anything; consequentialism: what actually is it? I have the sense that you’ve seen just as much of this phenomenon as I have, but you still seem to hold this conviction that there is a deep well of fundamental reasonability for an AI to fall into. Why? I’m just suggesting that the existence of this well is non-trivial feature of reality, and the more we fail to find it, the more we might question whether it exists.
In my own experience examining the foundations of things in the world, I have repeatedly found there to be less of an attractor-of-fundamentally-effective-decision-making than I had anticipated. In every way that I expected to find such an attractor within epistemology, mathematics, empiricism, ethics, I found in fact that even the very basic assumptions that I started with were unfounded, and found nothing firm to replace with them with. Probability theory: not a fundamental answer to epistemology; proof-based agents: deeply paradoxical and seemingly untrustworthy; the scientific method: not precise enough to be a final answer to anything; consequentialism: what actually is it? I have the sense that you’ve seen just as much of this phenomenon as I have, but you still seem to hold this conviction that there is a deep well of fundamental reasonability for an AI to fall into. Why? I’m just suggesting that the existence of this well is non-trivial feature of reality, and the more we fail to find it, the more we might question whether it exists.