I think that it would probably be a good idea to differentiate: ‘simple explanations’ and ‘explanations that are based on simple rules’. See Fake Simplicity for a description of simple explanations. An example would be attributing all of the causality to some other entity, e.g. god. Explanations that are based on simple rules can sometimes also be easy to understand, but the way in which they are reached is rarely simple. They are grounded in extensive research and evidence.
Simple explanations can be dangerous because they are easy to believe. They are:
Easy to understand
Sometimes partly correct as they can be true some of the time even though they don’t describe the whole picture
Often overly broad so that they are hard to disprove
Etc.
I am just trying to saying that we should also be careful of simple explanations as well because they can be enticing. I would think that non-experts rarely have enough experience to reach explanations based on simple rules and will instead often just find simple explanations. This is because it is really hard or, perhaps, even impossible to find these simple rules without a lot of ground work. We often have to understand something intimately and deeply before we even begin to sense the undercurrent from the operation of these simple rules.
The world is strange. The whole universe is very strange, but you see when you look at the details that the rules of the game are very simple – the mechanical rules by which you can figure out exactly what is going to happen when the situation is simple. It is like a chess game. If you are in a corner with only a few pieces involved, you can work out exactly what is going to happen, and you can always do that when there are only a few pieces. And yet in the real game there are so many pieces that you can’t figure out what is going to happen – so there is a kind of hierarchy of different complexities. It is hard to believe. It is incredible! In fact, most people don’t believe that the behavior of, say, me is the result of lots and lots of atoms all obeying very simple rules and evolving into such a creature that a billion years of life has produced. There is such a lot in the world. There is so much distance between the fundamental rules and the final phenomena that it is almost unbelievable that the final variety of phenomena can come from such a steady operation of such simple rules.
For the Feynman quote, I don’t think Feynman was a programmer! And Chaos was late in his life. Incomprehensibly complex behaviour from simple rules is not surprising to me.
Feynman certainly wasn’t a professional programmer and I don’t believe he spent much time writing software, but he worked for a while as a consultant to Thinking Machines Corporation and gave a (published, rather good) set of Lectures on Computation.
So if he doesn’t sound like a programmer, it may be for reasons other than not being familiar with the insights that come from spending time making software.
(I don’t think Feynman had any more difficulty than you in believing that simple rules produce very complex behaviour. I think he was trying to express how most people feel about it.)
I thought I was reasonably comfortable with basic parallel programming… then I spent a few hours talking with a very smart woman who specialized in low level parallel programming(close to the silicon) and talking about the kind of things that can go wrong.
It’s like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.
Something like dealing with synchronization and locking instead of just “and then we send this command to the mapper, and a lot of executors work in parallel”?
No, I was comfortable with locking and the sort of stuff you’ll routinely see in high level languages where they lock away the complexity behind abstraction.
I’m talking about the “fun” of physical cores which few sane programmers touch.
Though most programmers outside of chip companies can’t touch those levels nowdays since even assembly is treated like a high-level language.
I had something of a similar experience writing software for a company that designed its own chips. I was writing in C, and occasionally, the programs had errors that I couldn’t debug. At that point I’d dive into the Verilog for the chip and see how it was all wired up. And often that would be the problem.
It’s a commonplace that the thing that determines the meaning of a computer program is itself a computer program. But it’s still weird to get bitten by that fact when you’re writing C.
The chip is a program, which determines the meaning of the machine code, which is a program produced by the C compiler, which is a program interpreting the program I’m writing. And whether you look at the chip as a compiled version of the verilog (compiled by another program, with silicon as the target) or as a logical program which you’re proving things about in your head, or as a program being interpreted by a simulator, that’s a lot of turtles.....
Perhaps, ‘laws’ would have been a better word than ‘rules’.
I was thinking of it more in terms of complexity. When things are looked at in isolation, it is much easier to see how the simple laws apply. But as things get more complex, we also need to figure out how the different systems interact and influence each other. This makes the simple laws harder to discern.
Simple systems have few components and their behavior is in all respects fully
understandable and predictable. An example would be a solid ball falling under the action of gravity through air. This simple system consists of the ball, the air, and the gravitational force. Here we usually assume a single ball, constant acceleration of gravity, a viscous drag on the ball, and Newton’s laws. When making these assumptions, we arrive at very useful answers. We did, however, neglect many aspects. If, for example, we would ask how the behavior changes when we go from one ball, to two, to three, or even more balls that fall close to each other, our “Simple System” assumption fails. It is not sufficient to generalize from one ball’s behavior to many. Instead we need to consider the interaction of the balls through their self‐generated vortices.
I thought I was reasonably comfortable with basic parallel programming… then I spent a few hours talking with a woman who specialized in low level parallel programming(close to the silicon) and the kind of things that can go wrong.
It’s like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.
Yes, I think that’s all really solid. I’m definitely going for a ‘simple explanation’ rather than ‘an explanation based on simple rules’. It hasn’t even occurred to me that I can describe what’s going on mathematically.
And one should be very careful indeed of such ideas, especially if they have human consequences. Which is why I mentioned people who were probably trying to do their best to make the world a better place, who would probably have not been too pleased with how their ideas turned out.
But I keep thinking about atomism. Democritus worked it out thousands of years ago, from simple observations: ‘By convention there is colour, by convention there is sweetness, in reality, only atoms and the void’.
The essence of the truth. What Feynman called the one fact that he would like to communicate to an ignorant civilization.
And yet it took two thousand years to prove. And the answer was found by looking carefully into a cup of tea (poetically speaking!).
I do wonder whether the Greeks would have worked it out, if they’d had a real go instead of just believing their a priori assumptions.
There are many such examples.
In my particular case (which I am not claiming is anything like as important, even if it is bang right!):
Easy to understand
I think so, but everyone seems to think it’s absurd
Sometimes partly correct as they can be true some of the time even though they don’t describe the whole picture
That’s what I’m claiming. We should take it seriously and have a look!
Often overly broad so that they are hard to disprove
A great danger. Luckily I’ve suggested a simple experiment that would refute the whole idea beyond doubt.
I think that it would probably be a good idea to differentiate: ‘simple explanations’ and ‘explanations that are based on simple rules’. See Fake Simplicity for a description of simple explanations. An example would be attributing all of the causality to some other entity, e.g. god. Explanations that are based on simple rules can sometimes also be easy to understand, but the way in which they are reached is rarely simple. They are grounded in extensive research and evidence.
Simple explanations can be dangerous because they are easy to believe. They are:
Easy to understand
Sometimes partly correct as they can be true some of the time even though they don’t describe the whole picture
Often overly broad so that they are hard to disprove
Etc.
I am just trying to saying that we should also be careful of simple explanations as well because they can be enticing. I would think that non-experts rarely have enough experience to reach explanations based on simple rules and will instead often just find simple explanations. This is because it is really hard or, perhaps, even impossible to find these simple rules without a lot of ground work. We often have to understand something intimately and deeply before we even begin to sense the undercurrent from the operation of these simple rules.
Here’s an extract from Feynman which is related:
For the Feynman quote, I don’t think Feynman was a programmer! And Chaos was late in his life. Incomprehensibly complex behaviour from simple rules is not surprising to me.
Feynman certainly wasn’t a professional programmer and I don’t believe he spent much time writing software, but he worked for a while as a consultant to Thinking Machines Corporation and gave a (published, rather good) set of Lectures on Computation.
So if he doesn’t sound like a programmer, it may be for reasons other than not being familiar with the insights that come from spending time making software.
(I don’t think Feynman had any more difficulty than you in believing that simple rules produce very complex behaviour. I think he was trying to express how most people feel about it.)
And Feynman was very good at making people understand him. His lectures are well known for that.
I thought I was reasonably comfortable with basic parallel programming… then I spent a few hours talking with a very smart woman who specialized in low level parallel programming(close to the silicon) and talking about the kind of things that can go wrong.
It’s like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.
Something like dealing with synchronization and locking instead of just “and then we send this command to the mapper, and a lot of executors work in parallel”?
No, I was comfortable with locking and the sort of stuff you’ll routinely see in high level languages where they lock away the complexity behind abstraction.
I’m talking about the “fun” of physical cores which few sane programmers touch.
Though most programmers outside of chip companies can’t touch those levels nowdays since even assembly is treated like a high-level language.
http://blog.erratasec.com/2015/03/x86-is-high-level-language.html#.VvQhiOaAnSg
I had something of a similar experience writing software for a company that designed its own chips. I was writing in C, and occasionally, the programs had errors that I couldn’t debug. At that point I’d dive into the Verilog for the chip and see how it was all wired up. And often that would be the problem.
It’s a commonplace that the thing that determines the meaning of a computer program is itself a computer program. But it’s still weird to get bitten by that fact when you’re writing C.
The chip is a program, which determines the meaning of the machine code, which is a program produced by the C compiler, which is a program interpreting the program I’m writing. And whether you look at the chip as a compiled version of the verilog (compiled by another program, with silicon as the target) or as a logical program which you’re proving things about in your head, or as a program being interpreted by a simulator, that’s a lot of turtles.....
Perhaps, ‘laws’ would have been a better word than ‘rules’.
I was thinking of it more in terms of complexity. When things are looked at in isolation, it is much easier to see how the simple laws apply. But as things get more complex, we also need to figure out how the different systems interact and influence each other. This makes the simple laws harder to discern.
I thought I was reasonably comfortable with basic parallel programming… then I spent a few hours talking with a woman who specialized in low level parallel programming(close to the silicon) and the kind of things that can go wrong.
It’s like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.
Yes, I think that’s all really solid. I’m definitely going for a ‘simple explanation’ rather than ‘an explanation based on simple rules’. It hasn’t even occurred to me that I can describe what’s going on mathematically.
And one should be very careful indeed of such ideas, especially if they have human consequences. Which is why I mentioned people who were probably trying to do their best to make the world a better place, who would probably have not been too pleased with how their ideas turned out.
But I keep thinking about atomism. Democritus worked it out thousands of years ago, from simple observations: ‘By convention there is colour, by convention there is sweetness, in reality, only atoms and the void’.
The essence of the truth. What Feynman called the one fact that he would like to communicate to an ignorant civilization.
And yet it took two thousand years to prove. And the answer was found by looking carefully into a cup of tea (poetically speaking!).
I do wonder whether the Greeks would have worked it out, if they’d had a real go instead of just believing their a priori assumptions.
There are many such examples.
In my particular case (which I am not claiming is anything like as important, even if it is bang right!):
I think so, but everyone seems to think it’s absurd
That’s what I’m claiming. We should take it seriously and have a look!
A great danger. Luckily I’ve suggested a simple experiment that would refute the whole idea beyond doubt.