Reliability is way more important than people realized
Yes, but whence human reliability? What makes humans so much more reliable than the SotA AIs? What are AIs missing? The gulf in some cases is so vast it’s a quantity-is-a-quality-all-its-own thing.
1 is that the structure of jobs is shaped to accommodate human unreliability by making mistakes less fatal.
2 is that while humans themselves aren’t reliable, their algorithms almost certainly are more powerful at error detection and correction, so the big thing AI needs to achieve is the ability to error-correct or become more reliable.
There’s also the fact that humans are better at sample efficiency than most LLMs, but that’s a more debatable proposition.
the structure of jobs is shaped to accommodate human unreliability by making mistakes less fatal
Mm, so there’s a selection effect on the human end, where the only jobs/pursuits that exist are those which humans happen to be able to reliably do, and there’s a discrepancy between the things humans and AIs are reliable at, so we end up observing AIs being more unreliable, even though this isn’t representative of the average difference between the human vs. AI reliability across all possible tasks?
I don’t know that I buy this. Humans seem pretty decent at becoming reliable at ~anything, and I don’t think we’ve observed AIs being more-reliable-than-humans at anything? (Besides trivial and overly abstract tasks such as “next-token prediction”.)
My claim was more along the lines of if an unaided human can’t do a job safely or reliably, as was almost certainly the case 150-200 years ago, if not more years in the past, we make the jobs safer using tools such that human error is way less of a big deal, and AIs currently haven’t used tools that increased their reliability.
Remember, it took a long time for factories to be made safe, and I’d expect a similar outcome for driving, so while I don’t think 1 is everything, I do think it’s a non-trivial portion of the reliability difference.
Yes, but whence human reliability? What makes humans so much more reliable than the SotA AIs? What are AIs missing? The gulf in some cases is so vast it’s a quantity-is-a-quality-all-its-own thing.
I have 2 answers to this.
1 is that the structure of jobs is shaped to accommodate human unreliability by making mistakes less fatal.
2 is that while humans themselves aren’t reliable, their algorithms almost certainly are more powerful at error detection and correction, so the big thing AI needs to achieve is the ability to error-correct or become more reliable.
There’s also the fact that humans are better at sample efficiency than most LLMs, but that’s a more debatable proposition.
Mm, so there’s a selection effect on the human end, where the only jobs/pursuits that exist are those which humans happen to be able to reliably do, and there’s a discrepancy between the things humans and AIs are reliable at, so we end up observing AIs being more unreliable, even though this isn’t representative of the average difference between the human vs. AI reliability across all possible tasks?
I don’t know that I buy this. Humans seem pretty decent at becoming reliable at ~anything, and I don’t think we’ve observed AIs being more-reliable-than-humans at anything? (Besides trivial and overly abstract tasks such as “next-token prediction”.)
(2) seems more plausible to me.
My claim was more along the lines of if an unaided human can’t do a job safely or reliably, as was almost certainly the case 150-200 years ago, if not more years in the past, we make the jobs safer using tools such that human error is way less of a big deal, and AIs currently haven’t used tools that increased their reliability.
Remember, it took a long time for factories to be made safe, and I’d expect a similar outcome for driving, so while I don’t think 1 is everything, I do think it’s a non-trivial portion of the reliability difference.
More here:
https://www.lesswrong.com/posts/DQKgYhEYP86PLW7tZ/how-factories-were-made-safe