You could get to superintelligence that way, except that before that happens, someone else is going to make an AI that actively seeks out information and navigates the real world.
And it’s not all that safe in an absolute sense—large sequence models are so trustworthy specifically because we’re using them on problems where we can give lots of examples of humans solving them. By default, when you ask a big Q&A AI how to solve alignment, it will just tell you the sort of bad answer a human would give. Trying to avoid that default carries risks, and just seems like the wrong thing to be doing. Building tools to help humans solve the problem isn’t crazy, but this is different than expecting the answer to spring fully formed from a big AI that you trust without knowing much about alignment.
You could get to superintelligence that way, except that before that happens, someone else is going to make an AI that actively seeks out information and navigates the real world.
And it’s not all that safe in an absolute sense—large sequence models are so trustworthy specifically because we’re using them on problems where we can give lots of examples of humans solving them. By default, when you ask a big Q&A AI how to solve alignment, it will just tell you the sort of bad answer a human would give. Trying to avoid that default carries risks, and just seems like the wrong thing to be doing. Building tools to help humans solve the problem isn’t crazy, but this is different than expecting the answer to spring fully formed from a big AI that you trust without knowing much about alignment.
Thank you.