Traffic works well because driving is one of the activities that is relatively easy to internalize and perform as a matter of habit, like walking or riding a bicycle. System 1 thinking is usually fast, predictable and reliable. You don’t need to “focus intently”. It is true that you cannot take your eyes off the road (or, more accurately, shift your passive attention (see http://www.cdl.org/resource-library/articles/attention2.php) away) for longer than a brief instant, otherwise the subconscious feedback loop which includes visual and auditory inputs, your brain, your extremities and the vehicle breaks down, not because you have to consciously “focus intently”. The “magnets in the road” are built into the area of your brain that controls your reflexes. That’s why learning safe driving habits is so important.
Not everyone is equally capable of internalizing the driving process, or put in enough time to do so. These people white-knuckle it, constantly engaging their full slow and unsuitable System 2 in the loop, and consequently they find the normal driving activity exhausting, rather than relaxing. It is often easy to notice these drivers by their slow reaction to traffic lights, overly careful driving style and by generally overcompensating for road hazards. After all, System 2 is taxing, slow and unreliable. Ironically, these people may be safer drivers overall, since they never let their attention wander.
This passive attention escalates to active when something unusual happens, like when you hear emergency vehicles and have to make decisions which are not internalized. Or when some other car wanders into your lane, or if the traffic stops unexpectedly. Anything that either breaks the passive attention loop or causes this escalation from passive to active attention to slip is dangerous: texting, arguing, listening to a child fussing in the back, concentrating on a phone call.
In contrast, there are activities which do not naturally lend themselves to internalizing. Programming is one of them. Even after doing it for decades, people are still as consciously engaged in it as they did in the beginning. Thus your hope for a safe AGI “by analogy with safe driving”, seems misplaced to me.
Programming is one of them. Even after doing it for decades, people are still as consciously engaged in it as they did in the beginning.
My experience disagrees with this. After about 20 years of experience with C/C++, I have internalized many of the aspects of programming in this language, which allows me to write complex software factors of magnitude faster than 20 years ago, and factors of magnitude more safely.
I notice how much I have internalized when I switch to a different language that isn’t “my own”, and find myself immediately bogged down in all sorts of details for which I don’t know how exactly they work, and what is the best way to approach them.
In my experience, programming skill, especially in a particular language, does get internalized, much like dancing.
Maybe “still as consciously engaged in it as [...] in the beginning” was too strong, but compare programming to driving. If you’re like most drivers you can do basically anything while driving (in normal traffic and weather conditions), as long as it doesn’t require you to take your limbs from the car’s controls or your eyes from the road.
The programming equivalent of this would be that you can write a program (let’s say a binary tree implementation in C for the sake of argument) while having a conversation, and
this would not make you take noticeably longer to write the program
nor would it mess up your conversational ability.
A test of messing up your conversational ability would be whether the friend you’re talking to could tell over the phone that you’re doing something else at the same time.
(I’m guessing you can’t do this but I’d be interested to hear otherwise.)
To some extent, I think what you described does happen for snippets of code that are largely the same, and which one might write all the time. For example, I can write a “Hello world” program while maintaining conversation. However, as soon as you ask me to write something new, then I do have to start thinking about how to put pieces together, and can’t continue conversation.
But this also happens with driving. Speaking for myself at least, I can only maintain conversation while driving in a way that does not require me to make any decisions: (1) a route I’ve driven many times before, (2) a straight piece of road that might be unfamiliar, but does not require making any decisions.
If you put me in a new city where I don’t know where the streets are and how the traffic works, my conversational ability is much decreased (unless sitting at a red light, and perhaps even then, if I’m wondering where to turn next).
Programming tends to be like driving in new cities all the time. The difference we observe is really that we do most of our driving as a chore (same route, similar conditions each time) whereas we usually try to avoid that in programming (re-writing code we’ve already written several times, in similar conditions each time).
When you get good at driving, attention is freed for other things. When you get good at programming, the freed ressources you gain are used to speed up the programming. This speedup would, I think, not rollback without at least conscious effort and no other tasks can be attempted. The same should go for other skills whose effectiveness does/does not speed up with paid attention. If your desk job consisted of copying printed numbers into spreadsheets, you could probably divert attention after your input capacity is reached. If you were to juggle always as many balls as possible, you should never be distracted.
These people white-knuckle it, constantly engaging their full slow and unsuitable System 2 in the loop, and consequently they find the normal driving activity exhausting, rather than relaxing
There’s some of that in me. I probably am an overcautious driver.
Thus your hope for a safe AGI .. seems misplaced
Fair enough. Your regularly scheduled doom and gloom will resume shortly.
Here is my understanding of the issue.
Traffic works well because driving is one of the activities that is relatively easy to internalize and perform as a matter of habit, like walking or riding a bicycle. System 1 thinking is usually fast, predictable and reliable. You don’t need to “focus intently”. It is true that you cannot take your eyes off the road (or, more accurately, shift your passive attention (see http://www.cdl.org/resource-library/articles/attention2.php) away) for longer than a brief instant, otherwise the subconscious feedback loop which includes visual and auditory inputs, your brain, your extremities and the vehicle breaks down, not because you have to consciously “focus intently”. The “magnets in the road” are built into the area of your brain that controls your reflexes. That’s why learning safe driving habits is so important.
Not everyone is equally capable of internalizing the driving process, or put in enough time to do so. These people white-knuckle it, constantly engaging their full slow and unsuitable System 2 in the loop, and consequently they find the normal driving activity exhausting, rather than relaxing. It is often easy to notice these drivers by their slow reaction to traffic lights, overly careful driving style and by generally overcompensating for road hazards. After all, System 2 is taxing, slow and unreliable. Ironically, these people may be safer drivers overall, since they never let their attention wander.
This passive attention escalates to active when something unusual happens, like when you hear emergency vehicles and have to make decisions which are not internalized. Or when some other car wanders into your lane, or if the traffic stops unexpectedly. Anything that either breaks the passive attention loop or causes this escalation from passive to active attention to slip is dangerous: texting, arguing, listening to a child fussing in the back, concentrating on a phone call.
In contrast, there are activities which do not naturally lend themselves to internalizing. Programming is one of them. Even after doing it for decades, people are still as consciously engaged in it as they did in the beginning. Thus your hope for a safe AGI “by analogy with safe driving”, seems misplaced to me.
My experience disagrees with this. After about 20 years of experience with C/C++, I have internalized many of the aspects of programming in this language, which allows me to write complex software factors of magnitude faster than 20 years ago, and factors of magnitude more safely.
I notice how much I have internalized when I switch to a different language that isn’t “my own”, and find myself immediately bogged down in all sorts of details for which I don’t know how exactly they work, and what is the best way to approach them.
In my experience, programming skill, especially in a particular language, does get internalized, much like dancing.
Maybe “still as consciously engaged in it as [...] in the beginning” was too strong, but compare programming to driving. If you’re like most drivers you can do basically anything while driving (in normal traffic and weather conditions), as long as it doesn’t require you to take your limbs from the car’s controls or your eyes from the road.
The programming equivalent of this would be that you can write a program (let’s say a binary tree implementation in C for the sake of argument) while having a conversation, and
this would not make you take noticeably longer to write the program
nor would it mess up your conversational ability.
A test of messing up your conversational ability would be whether the friend you’re talking to could tell over the phone that you’re doing something else at the same time.
(I’m guessing you can’t do this but I’d be interested to hear otherwise.)
Sorry, I don’t check this place often.
To some extent, I think what you described does happen for snippets of code that are largely the same, and which one might write all the time. For example, I can write a “Hello world” program while maintaining conversation. However, as soon as you ask me to write something new, then I do have to start thinking about how to put pieces together, and can’t continue conversation.
But this also happens with driving. Speaking for myself at least, I can only maintain conversation while driving in a way that does not require me to make any decisions: (1) a route I’ve driven many times before, (2) a straight piece of road that might be unfamiliar, but does not require making any decisions.
If you put me in a new city where I don’t know where the streets are and how the traffic works, my conversational ability is much decreased (unless sitting at a red light, and perhaps even then, if I’m wondering where to turn next).
Programming tends to be like driving in new cities all the time. The difference we observe is really that we do most of our driving as a chore (same route, similar conditions each time) whereas we usually try to avoid that in programming (re-writing code we’ve already written several times, in similar conditions each time).
When you get good at driving, attention is freed for other things. When you get good at programming, the freed ressources you gain are used to speed up the programming. This speedup would, I think, not rollback without at least conscious effort and no other tasks can be attempted. The same should go for other skills whose effectiveness does/does not speed up with paid attention. If your desk job consisted of copying printed numbers into spreadsheets, you could probably divert attention after your input capacity is reached. If you were to juggle always as many balls as possible, you should never be distracted.
There’s some of that in me. I probably am an overcautious driver.
Fair enough. Your regularly scheduled doom and gloom will resume shortly.