What’s the problem with oracle AIs? It seems like if you had a safe oracle AI that gave human-aligned answers to questions, you could then ask “how do I make an aligned AGI?” and just do whatever it says. So it seems like the problem of “how to make an aligned agentic AGI” is no harder than “how to make an aligned orcale AI”, which I understand to still be extremely hard, but surely it’s easier than making an aligned agentic AGI from scratch?
my understanding is that while an oracle doesn’t directly control the nukes, it provides info to the people who control the nukes. Which is pretty much just moving the problem one layer deeper. While it can’t directly change the physical state of the world, it can manipulate people to pretty much achieve the same thing.
What’s the problem with oracle AIs? It seems like if you had a safe oracle AI that gave human-aligned answers to questions, you could then ask “how do I make an aligned AGI?” and just do whatever it says. So it seems like the problem of “how to make an aligned agentic AGI” is no harder than “how to make an aligned orcale AI”, which I understand to still be extremely hard, but surely it’s easier than making an aligned agentic AGI from scratch?
my understanding is that while an oracle doesn’t directly control the nukes, it provides info to the people who control the nukes. Which is pretty much just moving the problem one layer deeper. While it can’t directly change the physical state of the world, it can manipulate people to pretty much achieve the same thing.
Check this tag for more specifics: https://www.lesswrong.com/tag/oracle-ai