But a bigger part of the problem is that “I” don’t start them.
They just… get started. It’s like I’m infected with one of those old school viruses:
I’d dispute this. The thing that just happens is the new thread presenting itself. Opening it is another step. To quote Sam Harris, “one way to get rid of [the desire to check your emails] is to check your emails. Another is to notice the desire to check your emails as a pattern of energy and just let go.”
He tends to describe the human mind in its default state as helplessly bouncing from thought to thought, which to me sounds very similar to your description. But this has a pretty logical remedy in mindfulness.
Also, nitpick:
That is actually just an illusion though. In reality, the computer is spending a few units of time on Spotify, then a few on the file download, then a few on Zoom, then a few on the music again.
Isn’t this false nowadays that everyone has multi-core GPUs?
Great point about Sam Harris. I actually went through his meditation course Waking Up. I wanted to mention this point you make in the article but didn’t for whatever reason. In retrospect, I think I should have.
However, I think that there is a very All Therapy Books-y thing going on with meditation. In Scott’s article he talks about how most therapy books essentially claim that they’ve “figured it out”. But if that were true, why do so many patients continue to suffer? I have the same objections to meditation.
Isn’t this false nowadays that everyone has multi-core GPUs?
Nope, still applies. Even if you have more cores than running threads (remember programs are multi-threaded nowadays) and your OS could just hand one or more cores over indefinitely, it’ll generally still do a regular context switch to the OS and back several times per second.
And another thing that’s not worth its own comment but puts some numbers on the fuzzy “rapidly” from the article:
It’s just that [the process switching] happens rapidly.
For Windows, that’s traditionally 100 Hz, i. e. 100 context switches per second. For Linux it’s a kernel config parameter and you can choose a bunch of options between 100 and 1000 Hz. And “system calls” (e.g. talking to other programs or hardware like network / disk / sound card) can make those switches happen much more often.
There are no processes that can run independently on every time scale. There will be many clock cycles where every core is processing, and many where some cores are waiting on some shared resource to be available. Likewise if you look at parallelization via distinct hosts—they’re CLEARLY parallel, but only until they need data or instructions from outside.
The question for this post is “how much loss is there to wait times (both context switches and i/o waits), compared with some other way of organizing”? Primarily, the optimizations that are possible are around ensuring that the units of work are the right size to minimize the overhead of unnecessary synchronization points or wait times.
Nope, still applies. Even if you have more cores than running threads (remember programs are multi-threaded nowadays) and your OS could just hand one or more cores over indefinitely, it’ll generally still do a regular context switch to the OS and back several times per second.
Interesting. But does this mean “no two tasks are ever executed truly parallel-y” or just “we have true parallel execution but nonetheless have frequent context switches?”
I’d dispute this. The thing that just happens is the new thread presenting itself. Opening it is another step. To quote Sam Harris, “one way to get rid of [the desire to check your emails] is to check your emails. Another is to notice the desire to check your emails as a pattern of energy and just let go.”
He tends to describe the human mind in its default state as helplessly bouncing from thought to thought, which to me sounds very similar to your description. But this has a pretty logical remedy in mindfulness.
Also, nitpick:
Isn’t this false nowadays that everyone has multi-core GPUs?
Great point about Sam Harris. I actually went through his meditation course Waking Up. I wanted to mention this point you make in the article but didn’t for whatever reason. In retrospect, I think I should have.
However, I think that there is a very All Therapy Books-y thing going on with meditation. In Scott’s article he talks about how most therapy books essentially claim that they’ve “figured it out”. But if that were true, why do so many patients continue to suffer? I have the same objections to meditation.
Nope, still applies. Even if you have more cores than running threads (remember programs are multi-threaded nowadays) and your OS could just hand one or more cores over indefinitely, it’ll generally still do a regular context switch to the OS and back several times per second.
And another thing that’s not worth its own comment but puts some numbers on the fuzzy “rapidly” from the article:
For Windows, that’s traditionally 100 Hz, i. e. 100 context switches per second. For Linux it’s a kernel config parameter and you can choose a bunch of options between 100 and 1000 Hz. And “system calls” (e.g. talking to other programs or hardware like network / disk / sound card) can make those switches happen much more often.
There are no processes that can run independently on every time scale. There will be many clock cycles where every core is processing, and many where some cores are waiting on some shared resource to be available. Likewise if you look at parallelization via distinct hosts—they’re CLEARLY parallel, but only until they need data or instructions from outside.
The question for this post is “how much loss is there to wait times (both context switches and i/o waits), compared with some other way of organizing”? Primarily, the optimizations that are possible are around ensuring that the units of work are the right size to minimize the overhead of unnecessary synchronization points or wait times.
Interesting. But does this mean “no two tasks are ever executed truly parallel-y” or just “we have true parallel execution but nonetheless have frequent context switches?”
The latter. If you have 8 or 16 cores, it’d be really sad if only one thing was happening at a time.