Most patient cases are so easy and common that filling forms for an AI would greatly slow the system down.
I am assuming you’re not filling out any forms specially for the AI—just that the record-keeping system is computerized and the AI has access to it. In trivial cases the AI won’t have much data (e.g. no fever, normal blood pressure, complains of a running nose and cough, that’s it) and its diagnoses will be low-credence, but that’s fine, you as a doctor won’t need its assistance in those cases.
The AI would need to know natural language to be of any use or else it will miss most of the relevant data. I suppose Watson is pretty close to that and have read that it’s tested in some hospitals. I wonder how this is implemented. I suspect doctors carry a lot more data in their heads than is readily apparent and much of this data will never make it to their notes and thus to the computerized records.
Taking a history, evaluating it’s reliability and using the senses to observe the patients are something machines can’t do for quite some time. On top of this I roughly know hundreds of patients now that I will see time and again and this helps immensely when judging their most acute presentations. By this I don’t mean I know them as lists of symptoms, but I know their personalities too and how this affects how they tell their stories and how seriously they take their symptoms from minor complaints to major problems. I could never take the approach of jumping from a hospital to hospital now that I’ve experienced this first hand.
The AI would need to know natural language to be of any use or else it will miss most of the relevant data. I suppose Watson is pretty close to that and have read that it’s tested in some hospitals. I wonder how this is implemented. I suspect doctors carry a lot more data in their heads than is readily apparent and much of this data will never make it to their notes and thus to the computerized records.
This is the reason Watson is a game-changer, despite expert prediction systems (using linear regression!) performing at the level of expert humans for ~50 years. Doctors may carry a lot of information in their heads, but I’ve yet to meet a person that’s able to mentally invert matrices of non-trivial size, which helps quite a bit with determining the underlying structure of the data and how best to use it.
Taking a history, evaluating it’s reliability and using the senses to observe the patients are something machines can’t do for quite some time.
I think machines have several comparative advantages here. An AI with basic conversational functions can take a history, and is better at evaluating some parts of the reliability and worse at others. It can compare with ‘other physicians’ more easily, or check public records, but probably can’t determine whether or not it’s a coherent narrative as easily (“What is Toronto?”). A webcam can measure pulse rate just by looking, and so I suspect it’ll be about as good at detecting deflection and lying as the average doctor. (I don’t remember seeing doctors as being particularly good at lie-detection, but it’s been a while since I’ve read any of the lie-detection literature.)
I could never take the approach of jumping from a hospital to hospital now that I’ve experienced this first hand.
Note that if the AI is sufficiently broadly used (here I’m imagining, say, the NHS in the UK using just one) then everyone will always have access to a doctor that’s known them as long as they’ve been in the system.
despite expert prediction systems (using linear regression!) performing at the level of expert humans for ~50 years.
Is this because using them is incredibly slow or something else?
A webcam can measure pulse rate just by looking, and so I suspect it’ll be about as good at detecting deflection and lying as the average doctor. (I don’t remember seeing doctors as being particularly good at lie-detection, but it’s been a while since I’ve read any of the lie-detection literature.)
Lies make no sense medically, or make too much sense. Once I’ve spotted a few lies, many of them fit a stereotypical pattern many patients use even if there aren’t any other clues. I don’t need to rely on body language much.
People also misremember things, or have a helpful relative misremember things for them, or home care providers feeding their clueless preliminary diagnoses for these people. People who don’t remember fill in the gap with something they think is plausible. Some people are also psychotic or don’t even remember what year it is or why they came in the first place. Some people treat every little ache like it’s the end of the world and some don’t seem to care if their leg’s missing.
I think even an independent AI could make up for many of its faults simply by being more accurate at interpreting the records and current test results.
I hope that when an AI can do my job I don’t need a job anymore :)
Is this because using them is incredibly slow or something else?
My understanding is that the ~4 measurements the system would use as inputs were typically measured by the doctor, and by the time the doctor had collected the data they had simultaneously come up with their own diagnosis. Typing the observations into the computer to get the same level of accuracy (or a few extra percentage points) rarely seemed worth it, and turning the doctor from a diagnostician to a tech was, to put it lightly, not popular with doctors. :P
There are other arguments which would take a long time to go into. One is “but what about X?”, where the linear regression wouldn’t take into account some other variable that the human could take into account, and so the human would want an override option. But, as one might expect, the only way for the regression to outperform the human is for the regression to be right more often than not when the two of them disagree, and humans are unfortunately not very good at determining whether or not the case in front of them is a special case where an override will increase accuracy or a normal case where an override will decrease accuracy. Here’s probably the best place to start if interested in reading more.
A rather limited subset of the natural language, I think it’s a surmountable problem.
I suspect doctors carry a lot more data in their heads than is readily apparent … I roughly know hundreds of patients now that I will see time and again and this helps immensely when judging their most acute presentations.
All true, which is why I think a well-designed diagnostic AI will work in partnership with a doctor instead of replacing him.
I agree with you, but I fear that makes for a boring conversation :)
The language is already relatively standardized and I suppose you could standardize it more to make it easier for the AI. I suspect any attempt to mold the system for an AI would meet heavy resistance however.
I am assuming you’re not filling out any forms specially for the AI—just that the record-keeping system is computerized and the AI has access to it. In trivial cases the AI won’t have much data (e.g. no fever, normal blood pressure, complains of a running nose and cough, that’s it) and its diagnoses will be low-credence, but that’s fine, you as a doctor won’t need its assistance in those cases.
The AI would need to know natural language to be of any use or else it will miss most of the relevant data. I suppose Watson is pretty close to that and have read that it’s tested in some hospitals. I wonder how this is implemented. I suspect doctors carry a lot more data in their heads than is readily apparent and much of this data will never make it to their notes and thus to the computerized records.
Taking a history, evaluating it’s reliability and using the senses to observe the patients are something machines can’t do for quite some time. On top of this I roughly know hundreds of patients now that I will see time and again and this helps immensely when judging their most acute presentations. By this I don’t mean I know them as lists of symptoms, but I know their personalities too and how this affects how they tell their stories and how seriously they take their symptoms from minor complaints to major problems. I could never take the approach of jumping from a hospital to hospital now that I’ve experienced this first hand.
This is the reason Watson is a game-changer, despite expert prediction systems (using linear regression!) performing at the level of expert humans for ~50 years. Doctors may carry a lot of information in their heads, but I’ve yet to meet a person that’s able to mentally invert matrices of non-trivial size, which helps quite a bit with determining the underlying structure of the data and how best to use it.
I think machines have several comparative advantages here. An AI with basic conversational functions can take a history, and is better at evaluating some parts of the reliability and worse at others. It can compare with ‘other physicians’ more easily, or check public records, but probably can’t determine whether or not it’s a coherent narrative as easily (“What is Toronto?”). A webcam can measure pulse rate just by looking, and so I suspect it’ll be about as good at detecting deflection and lying as the average doctor. (I don’t remember seeing doctors as being particularly good at lie-detection, but it’s been a while since I’ve read any of the lie-detection literature.)
Note that if the AI is sufficiently broadly used (here I’m imagining, say, the NHS in the UK using just one) then everyone will always have access to a doctor that’s known them as long as they’ve been in the system.
Is this because using them is incredibly slow or something else?
Lies make no sense medically, or make too much sense. Once I’ve spotted a few lies, many of them fit a stereotypical pattern many patients use even if there aren’t any other clues. I don’t need to rely on body language much.
People also misremember things, or have a helpful relative misremember things for them, or home care providers feeding their clueless preliminary diagnoses for these people. People who don’t remember fill in the gap with something they think is plausible. Some people are also psychotic or don’t even remember what year it is or why they came in the first place. Some people treat every little ache like it’s the end of the world and some don’t seem to care if their leg’s missing.
I think even an independent AI could make up for many of its faults simply by being more accurate at interpreting the records and current test results.
I hope that when an AI can do my job I don’t need a job anymore :)
My understanding is that the ~4 measurements the system would use as inputs were typically measured by the doctor, and by the time the doctor had collected the data they had simultaneously come up with their own diagnosis. Typing the observations into the computer to get the same level of accuracy (or a few extra percentage points) rarely seemed worth it, and turning the doctor from a diagnostician to a tech was, to put it lightly, not popular with doctors. :P
There are other arguments which would take a long time to go into. One is “but what about X?”, where the linear regression wouldn’t take into account some other variable that the human could take into account, and so the human would want an override option. But, as one might expect, the only way for the regression to outperform the human is for the regression to be right more often than not when the two of them disagree, and humans are unfortunately not very good at determining whether or not the case in front of them is a special case where an override will increase accuracy or a normal case where an override will decrease accuracy. Here’s probably the best place to start if interested in reading more.
A rather limited subset of the natural language, I think it’s a surmountable problem.
All true, which is why I think a well-designed diagnostic AI will work in partnership with a doctor instead of replacing him.
I agree with you, but I fear that makes for a boring conversation :)
The language is already relatively standardized and I suppose you could standardize it more to make it easier for the AI. I suspect any attempt to mold the system for an AI would meet heavy resistance however.