1) It’s possible to choose a basis which does a good job separating the slope from the level
2) Our perturbations are all small relative to the curvature of the terrain, such that we can model things as an n-dimensional plane
3) “Known” errors can be easily avoided, even in many dimensional space, such that the main remaining question is what the right answers are
4) Maintenance of higher standards doesn’t help distinguish between better and worse directions.
5) Drama pushes in random directions, rather than directions selected for being important and easy to fuck up.
1) In a high dimensional space, almost all bases have the slope distributed among many basis vectors. If you can find a basis that has a basis vector pointing right down the gradient and the rest normal to it, that’s great. If your bridge has one weak strut, fix it. However, there’s no reason to suspect we can always or even usually do this. If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there’s no way you could do it simply. You could name the direction “more better”, but in order to actually point at it or build a bridge, many many design choices will have to be made. In most real world problems, you need to look in many individual directions and decide whether it’s an improvement or not and how far to go. Real world value is built on many “marginal” improvements.
2) The fact that we’re even breathing at all means that we’ve stacked up a lot of them. Almost every configuration is completely non-functional, and being in any way coherent requires getting a lot of things right. We are balanced near optima on many dimensions, even thought there is plenty left to go. While almost all “small” deviations have even smaller impact, almost all “large” deviations cause a regression to the mean or at least have more potential loss than gain. The question is whether all perturbations can be assumed small, and the answer is clear from looking at the estimated curvature. On a bad day you can easily exhibit half the tolerance that you do on a good day. Different social settings can change the tolerance by *much* more than that. I could be pretty easily convinced that I’m averaging 10% too tolerant or 10% too intolerant, but a factor of two either way is pretty clearly bad in expectation. In other words, the terrain is can *not* be taken as planar.
3) Going uphill, even when you know which way is up, is *hard*, and there is a tendency to downslide. Try losing weight, if you have any to lose. Try exercising as much as you think you should. Or just hiking up a real mountain. Gusts of wind don’t blow you up the mountain as often as they push you down; gusts of wind cause you to lose your footing, and when you lose your footing you inevitably degenerate into a high entropy mess that is further from the top. Getting too little sleep, or being yelled at too much, doesn’t cause people to do better as often as it causes them to do worse. It causes people to lose track of longer term consequences, and short term gradient following leads to bad long term results. This is because so many problems are non-minimum phase. Bike riding requires counter-steering. Strength training requires weight lifting, and accepting temporary weakening. Getting rewarded for clear thinking requires first confronting the mistakes you’ve been making. “Knowing which way to go” is an important part of the problem too, and it does become limiting once you get your other stuff in order, but “consistently performs as well as they could, given what they know” is a damn high bar, and we’re not there yet. “Do the damn things you know you’re supposed to to, and don’t rationalize excuses” is a really important part of it, and not as easy as it sounds.
4) Our progress on one dimension is not independent of our ability to progress on the others. Eat unhealthy foods despite knowing better, and you might lose a day of good mental performance that you could have use to figure out “which direction?”. Let yourself believe a comforting belief, and that little deviation from the truth can lead to much larger problems in the future. One of the coolest things about LW, in my view, is that people here are epistemically careful enough that they don’t shoot themselves in the foot *immediately*. Most people reason themselves into traps so quickly that you either have to be extremely careful with the order and manner in which you present things, or else you have to cultivate an unusual amount of respect so they’ll listen for long enough to notice their confusion. LW is *better* at this. LW is not *perfect* at this. More is better. We don’t have clear thinking to burn. So much of clear thinking has to do with having room to countersteer that doing anything but maximizing it to the best of our ability is a huge loss in future improvement.
5) Drama is not unimportant, and it is not separable. We are social creatures, and the health and direction of our social structures is a big deal. If you want to get anything done as a community, whether it be personal rationality improvement or collective efforts, the community has to function or that ain’t gonna happen. That involves a lot of discussing which norms and beliefs should be adopted, as well as meta-norms and beliefs about how disagreement should be handled, and applying to relevant cases. Problems with bad thinking become exposed and that makes such discussions both more difficult and more risky, but also more valuable to get right. Hubris that gets you in trouble when talking to others doesn’t just go away when making private plans and decisions, but in those cases you do lack someone to call you on it and therefore can’t so easily find which direction(s) you are erring in. Drama isn’t a “random distraction”, it’s an error signal showing that something is wrong with your/your communities sense making organs, and you need those things in order to find the right directions and then take them. It’s not the *only* thing, and there are plenty of ways to screw it up while thinking you’re doing the right thing (non-minimumphase again), but it is selected (if imperfectly) for being centered around the most important disagreements, or else it wouldn’t command the attention that it does.
This is a great comment. There are some parts which I think are outright wrong (e.g. drama selects for most-important-disagreements), but for the most part it correctly identifies a bunch of shortcomings of the linear model from my comment.
I do think these shortcomings can generally be patched; the linear model is just one way to explain the core idea, and other models lead to the same place. The main idea is something like “in a high dimensional space, choosing the right places to explore is way more important than speed of exploration”, and that generalizes well beyond linearity.
I’m not going to flesh that out more right now. This all deserves a better explanation than I’m currently ready to write.
Yeah, I anticipated that the “Drama is actually kinda important” bit would be somewhat controversial. I did qualify that it was selected “(if imperfectly)” :p
Most things are like “Do we buy our scratch paper from walmart or kinkos?”, and there are few messes of people so bad that it’d make me want to say “Hey, I know you think what you’re fighting about is important, but it’s literally less important than where we buy our scratch paper, whether we name our log files .log or .txt, and literally any other random thing you can think of”.
(Actually, now that I say this, I realize that it can fairly often look that way and that’s why “bikeshedding” is a term. I think those are complicated by factors like “What they appear to be fighting about isn’t really what they’re fighting about”, “Their goals aren’t aligned with the goal you’re measuring them relative to”, and “The relevant metric isn’t how well they can select on an absolute scale or relative to your ability, but relative to their own relatively meager abilities”.)
In one extreme, you say “Look, you’re fighting about this for a reason, it’s clearly the most important thing, or at least top five, ignore anyone arguing otherwise”.
In another, you say “Drama can be treated as random noise, and the actual things motivating conflict aren’t in any way significantly more important than any other randomly selected thing one could attend to, so the correct advice is just to ignore those impulses and plow forward”
I don’t think either are very good ways of doing it, to understate it a bit. “Is this really what’s important here?” is an important question to keep in mind (which people sometimes forget, hence point 3), but that it cannot be treated as a rhetorical question and must be asked in earnest because the answer can very well be “Yes, to the best of my ability to tell”—especially within groups of higher functioning individuals.
I think we do have a real substantive disagreement in that I think the ability to handle drama skillfully is more important and also more directly tied into more generalized rationality skills than you do, but that’s a big topic to get into.
I am, however, in full agreement on the main idea of “in a high dimensional space, choosing the right places to explore is way more important than speed of exploration”, and that it generalizes well and is a very important concept. It’s actually pretty amusing that I find myself arguing “the other side” here, given that so much of what I do for work (and otherwise) involves face palming about people working really hard to optimize the wrong part of the pie chart, instead of realizing to make a pie chart and work only on the biggest piece or few.
If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there’s no way you could do it simply. You could name the direction “more better”, but in order to actually point at it or build a bridge, many many design choices will have to be made. In most real world problems, you need to look in many individual directions and decide whether it’s an improvement or not and how far to go. Real world value is built on many “marginal” improvements.
This was an outstandingly useful mental image for me, and one I suspect I will incorporate into a lot of thoughts and explanations. Thanks.
EDIT: finished reading the rest of this, and it’s tied (with Vaniver’s) for my favorite comment on this post (at least as far as the object level is concerned; there are some really good comments about the discussions themselves).
I think your main point here is wrong.
Your analysis rests on a lot of assumptions:
1) It’s possible to choose a basis which does a good job separating the slope from the level
2) Our perturbations are all small relative to the curvature of the terrain, such that we can model things as an n-dimensional plane
3) “Known” errors can be easily avoided, even in many dimensional space, such that the main remaining question is what the right answers are
4) Maintenance of higher standards doesn’t help distinguish between better and worse directions.
5) Drama pushes in random directions, rather than directions selected for being important and easy to fuck up.
1) In a high dimensional space, almost all bases have the slope distributed among many basis vectors. If you can find a basis that has a basis vector pointing right down the gradient and the rest normal to it, that’s great. If your bridge has one weak strut, fix it. However, there’s no reason to suspect we can always or even usually do this. If you had to describe the direction of improvement from a rotting log to a nice cable stayed bridge, there’s no way you could do it simply. You could name the direction “more better”, but in order to actually point at it or build a bridge, many many design choices will have to be made. In most real world problems, you need to look in many individual directions and decide whether it’s an improvement or not and how far to go. Real world value is built on many “marginal” improvements.
2) The fact that we’re even breathing at all means that we’ve stacked up a lot of them. Almost every configuration is completely non-functional, and being in any way coherent requires getting a lot of things right. We are balanced near optima on many dimensions, even thought there is plenty left to go. While almost all “small” deviations have even smaller impact, almost all “large” deviations cause a regression to the mean or at least have more potential loss than gain. The question is whether all perturbations can be assumed small, and the answer is clear from looking at the estimated curvature. On a bad day you can easily exhibit half the tolerance that you do on a good day. Different social settings can change the tolerance by *much* more than that. I could be pretty easily convinced that I’m averaging 10% too tolerant or 10% too intolerant, but a factor of two either way is pretty clearly bad in expectation. In other words, the terrain is can *not* be taken as planar.
3) Going uphill, even when you know which way is up, is *hard*, and there is a tendency to downslide. Try losing weight, if you have any to lose. Try exercising as much as you think you should. Or just hiking up a real mountain. Gusts of wind don’t blow you up the mountain as often as they push you down; gusts of wind cause you to lose your footing, and when you lose your footing you inevitably degenerate into a high entropy mess that is further from the top. Getting too little sleep, or being yelled at too much, doesn’t cause people to do better as often as it causes them to do worse. It causes people to lose track of longer term consequences, and short term gradient following leads to bad long term results. This is because so many problems are non-minimum phase. Bike riding requires counter-steering. Strength training requires weight lifting, and accepting temporary weakening. Getting rewarded for clear thinking requires first confronting the mistakes you’ve been making. “Knowing which way to go” is an important part of the problem too, and it does become limiting once you get your other stuff in order, but “consistently performs as well as they could, given what they know” is a damn high bar, and we’re not there yet. “Do the damn things you know you’re supposed to to, and don’t rationalize excuses” is a really important part of it, and not as easy as it sounds.
4) Our progress on one dimension is not independent of our ability to progress on the others. Eat unhealthy foods despite knowing better, and you might lose a day of good mental performance that you could have use to figure out “which direction?”. Let yourself believe a comforting belief, and that little deviation from the truth can lead to much larger problems in the future. One of the coolest things about LW, in my view, is that people here are epistemically careful enough that they don’t shoot themselves in the foot *immediately*. Most people reason themselves into traps so quickly that you either have to be extremely careful with the order and manner in which you present things, or else you have to cultivate an unusual amount of respect so they’ll listen for long enough to notice their confusion. LW is *better* at this. LW is not *perfect* at this. More is better. We don’t have clear thinking to burn. So much of clear thinking has to do with having room to countersteer that doing anything but maximizing it to the best of our ability is a huge loss in future improvement.
5) Drama is not unimportant, and it is not separable. We are social creatures, and the health and direction of our social structures is a big deal. If you want to get anything done as a community, whether it be personal rationality improvement or collective efforts, the community has to function or that ain’t gonna happen. That involves a lot of discussing which norms and beliefs should be adopted, as well as meta-norms and beliefs about how disagreement should be handled, and applying to relevant cases. Problems with bad thinking become exposed and that makes such discussions both more difficult and more risky, but also more valuable to get right. Hubris that gets you in trouble when talking to others doesn’t just go away when making private plans and decisions, but in those cases you do lack someone to call you on it and therefore can’t so easily find which direction(s) you are erring in. Drama isn’t a “random distraction”, it’s an error signal showing that something is wrong with your/your communities sense making organs, and you need those things in order to find the right directions and then take them. It’s not the *only* thing, and there are plenty of ways to screw it up while thinking you’re doing the right thing (non-minimumphase again), but it is selected (if imperfectly) for being centered around the most important disagreements, or else it wouldn’t command the attention that it does.
This is a great comment. There are some parts which I think are outright wrong (e.g. drama selects for most-important-disagreements), but for the most part it correctly identifies a bunch of shortcomings of the linear model from my comment.
I do think these shortcomings can generally be patched; the linear model is just one way to explain the core idea, and other models lead to the same place. The main idea is something like “in a high dimensional space, choosing the right places to explore is way more important than speed of exploration”, and that generalizes well beyond linearity.
I’m not going to flesh that out more right now. This all deserves a better explanation than I’m currently ready to write.
Yeah, I anticipated that the “Drama is actually kinda important” bit would be somewhat controversial. I did qualify that it was selected “(if imperfectly)” :p
Most things are like “Do we buy our scratch paper from walmart or kinkos?”, and there are few messes of people so bad that it’d make me want to say “Hey, I know you think what you’re fighting about is important, but it’s literally less important than where we buy our scratch paper, whether we name our log files .log or .txt, and literally any other random thing you can think of”.
(Actually, now that I say this, I realize that it can fairly often look that way and that’s why “bikeshedding” is a term. I think those are complicated by factors like “What they appear to be fighting about isn’t really what they’re fighting about”, “Their goals aren’t aligned with the goal you’re measuring them relative to”, and “The relevant metric isn’t how well they can select on an absolute scale or relative to your ability, but relative to their own relatively meager abilities”.)
In one extreme, you say “Look, you’re fighting about this for a reason, it’s clearly the most important thing, or at least top five, ignore anyone arguing otherwise”.
In another, you say “Drama can be treated as random noise, and the actual things motivating conflict aren’t in any way significantly more important than any other randomly selected thing one could attend to, so the correct advice is just to ignore those impulses and plow forward”
I don’t think either are very good ways of doing it, to understate it a bit. “Is this really what’s important here?” is an important question to keep in mind (which people sometimes forget, hence point 3), but that it cannot be treated as a rhetorical question and must be asked in earnest because the answer can very well be “Yes, to the best of my ability to tell”—especially within groups of higher functioning individuals.
I think we do have a real substantive disagreement in that I think the ability to handle drama skillfully is more important and also more directly tied into more generalized rationality skills than you do, but that’s a big topic to get into.
I am, however, in full agreement on the main idea of “in a high dimensional space, choosing the right places to explore is way more important than speed of exploration”, and that it generalizes well and is a very important concept. It’s actually pretty amusing that I find myself arguing “the other side” here, given that so much of what I do for work (and otherwise) involves face palming about people working really hard to optimize the wrong part of the pie chart, instead of realizing to make a pie chart and work only on the biggest piece or few.
This was an outstandingly useful mental image for me, and one I suspect I will incorporate into a lot of thoughts and explanations. Thanks.
EDIT: finished reading the rest of this, and it’s tied (with Vaniver’s) for my favorite comment on this post (at least as far as the object level is concerned; there are some really good comments about the discussions themselves).
Outstanding comment. (Easily the best I’ve read on Less Wrong in the last month, top five in the last year.)