You seem to imply that medical AI would be a safer domain to invest in AI development than other domains, yet you offer few details about this. I know you say in the paper that it’s outside the scope of this work, but can you give a summary or an outline of your thoughts here? Right now it reads to me, rather unconvincingly, that somehow it will be safer because humans will be trying to use it for human-valued purposes, but this is contrary to the orthogonality thesis so this implies you have some reason to think the orthogonality thesis is, if not wrong, at least weaker than, say, MIRI presents it to be.
Yes, the editors asked me to cut my thoughts about domains specific effects in AI safety as they will be off topic in the already large paper. Also they suggested that form the point of view of “fooming” military AI will be safer than medical AI, as they militaries invest more in control of their systems, than startups of biohackers, and it seems to be true if fooming and orthogonality thesis hold.
However, domain specificity may affect the orthogonality thesis in the following way. If an agent has an instrument X, it may try to solve all its tasks by using this instrument. For example, - let’s take a human example—a businessman my pay for everything he wants, and a hitman will try to use violence for everything he wants. So where will be a correlation between available goals and available instruments.
In case of (stupid) AI, if you ask military AI to solve cancer problem, it will make a nuclear strike on all continents, but if you ask medical AI, it will create a cure for cancer. Will this hold for superintelligence is not currently known.
Another reason why medical AI will be safer is that it will be more integrated with human mind from the start, as it will be based around brain-computer interfaces, human uploads or different augmentation technics and because of this, it will be closer to human thinking process (so there will be no common sense understanding errors) and even may be to human values, if humans will be at its core, as it could happen if medical AI produses hansonian ems world, and superintelligence will appear as augmentation of uploads, or as a result of collective behaviour of many augmented uploads. I am going to explore all these hypothetical scenarios in another paper for which I have an early draft. (edit: grammar)
You seem to imply that medical AI would be a safer domain to invest in AI development than other domains, yet you offer few details about this. I know you say in the paper that it’s outside the scope of this work, but can you give a summary or an outline of your thoughts here? Right now it reads to me, rather unconvincingly, that somehow it will be safer because humans will be trying to use it for human-valued purposes, but this is contrary to the orthogonality thesis so this implies you have some reason to think the orthogonality thesis is, if not wrong, at least weaker than, say, MIRI presents it to be.
Yes, the editors asked me to cut my thoughts about domains specific effects in AI safety as they will be off topic in the already large paper. Also they suggested that form the point of view of “fooming” military AI will be safer than medical AI, as they militaries invest more in control of their systems, than startups of biohackers, and it seems to be true if fooming and orthogonality thesis hold.
However, domain specificity may affect the orthogonality thesis in the following way. If an agent has an instrument X, it may try to solve all its tasks by using this instrument. For example, - let’s take a human example—a businessman my pay for everything he wants, and a hitman will try to use violence for everything he wants. So where will be a correlation between available goals and available instruments.
In case of (stupid) AI, if you ask military AI to solve cancer problem, it will make a nuclear strike on all continents, but if you ask medical AI, it will create a cure for cancer. Will this hold for superintelligence is not currently known.
Another reason why medical AI will be safer is that it will be more integrated with human mind from the start, as it will be based around brain-computer interfaces, human uploads or different augmentation technics and because of this, it will be closer to human thinking process (so there will be no common sense understanding errors) and even may be to human values, if humans will be at its core, as it could happen if medical AI produses hansonian ems world, and superintelligence will appear as augmentation of uploads, or as a result of collective behaviour of many augmented uploads. I am going to explore all these hypothetical scenarios in another paper for which I have an early draft. (edit: grammar)
Great! I look forward to hearing about such ideas in more detail.