This article is quite long. As general feedback, I won’t usually bother reading long articles unless they summarise their content up front with an abstract, or something similar. This post starts with more of a teaser. A synopsis at the end would be good as well: tell me three times.
An observation: PJeby if you really have a self help product that does what it says on the tin for anyone who gives it a fair try, I would argue that the most efficient way of establishing credibility among the Less wrong community would be to convince a highly regarded poster of that fact. To that end I would suggest that offering your product to Eliezer Yudkowsky for free or even paying him to try it in the form of a donation to his singularity institute would be more effective than the back and forth that I see here.
It should be possible to establish an mutually satisfactory set of criteria of what constitutes ‘really trying it’ beforehand to avoid subsequent accusations of bad faith.
I would argue that the most efficient way of establishing credibility among the Less wrong community would be to convince a highly regarded poster of that fact.
Pjeby: If your goal isn’t to convince the less wrong community of the effectiveness of your methodology then I am truly puzzled as to why you post here. If convincing others is not your goal, then what is?
One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
I said “one does not have to be convinced of the correctness of something in order to test it”, and you replied with something about usefulness. Therefore, there is nothing I can say about your response except that it’s utterly unrelated to what I said.
You are the one who introduced correctness into the argument. Alicorn said:
Do you expect anyone to benefit from your expertise if you can’t convince them you have it?
Feel free to read this as ‘convince them your expertise is “useful” ’ rather than your assumed ‘convince them your expertise is “correct” ’.
The underlying point is that there is a very large amount of apparently useless advice out there, and many self-help techniques seem initially useful but then stop being useful. (as you are well aware since your theory claims to explain why it happens)
The problem is to convince someone to try your advice, you have to convince them that the (probability of it being useful claimed benefit probability of claim being correct) is greater than the opportunity cost of the expected effort to try it. Due to others in the self-help market, the prior for it being useful is very low, and the prior for the claimed benefits equaling the actual benefits is low.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale. Dishonest salespeople spoil the sales possibilities for all the honest ones.
If you can convince someone with a higher standing in the community than you to test your advice and comment on the results of their test, you can raise individual’s probability expectations about the usefulness (or correctness) of your advice, and hence help more people than you otherwise would have.
P.S. I did go to your site and get added to your mailing list. However, even if your techniques turn out positively for me, I don’t think I have any higher standing in this community than you do, so I doubt my results will hold much weight with this group.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale.
Actually, I’m also running into a bias that merely because I have things to sell, I’m therefore trying to sell something in all places at all times… or that I’m always trying to “convince” people of something.
Indeed, the fact that you (and others) seem to think I need or even want to “convince” people of things is a symptom of this. Nobody goes around insisting that say, Yvain needs to get some high-status people to validate his ideas and “convince” the “community” to accept them!
If I had it all to do over again, I think I would have joined under a pseudonym and never let on I even had a business.
You are certainly right that “one does not have to be convinced of the correctness of something in order to test it.” But as you also said, immediately prior, “Either someone uses the information I give or they don’t.”
If we test information that we do not have reason to believe is useful, then we have a massive search space to cover. Much of the point of LW is to suggest useful regions for search, based on previous data.
So no, correctness is not a necessary condition of usefulness. But things that are correct are usually rather useful, and things that are not correct are less so. To the extent that you or your expertise are reliable indicators of the quality of your information, they help evaluate the probability of your information being useful, and hence the expected benefit of testing it.
Perhaps some parties on LW are actually confused by the distinction between truth and utility. I do not suspect Vladimir_Nesov is one of them.
But things that are correct are usually rather useful, and things that are not correct are less so.
Really? With what probability?
Or to put it another way: how were people were to start and put out fires for millennia before they had a correct theory of fire? Work metals without a correct atomic or molecular theory? Build catapults without a correct theory of gravity? Breed plants and animals without a correct theory of genetics?
In the entire history of humanity, “Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”, but there’s usually a ton more information you need besides the correct theory to get to “useful”, and more often, the theory ends up being derived from something that’s already “useful” anyway.
That’s a shockingly poor argument. Who can constrain the future more effectively: someone who knows the thermodynamics of combustion engines, or someone who only knows how to start fires with a flint-and-steel and how to stop them with water? Someone who can use X-ray crystallography to assess their metallurgy, or someone who has to whack their product with a mallet to see if it’s brittle? Someone who can fire mortars over ranges requiring Coriolis corrections (i.e., someone with a correct theory of mechanics) or someone who only knows how to aim a catapult by trial and error? Someone who can insert and delete bacterial genes, or someone who doesn’t even know germ theory?
Someone who actually knows how human cognition works on all scales, or someone with the equivalent of a set of flint-and-steel level tools and a devotion to trial and error?
‘Correctness’ in theories is a scalar rather than a binary quality. Phlogiston theory is less correct (and less useful) than chemistry, but it’s more correct—and more useful!--than the theory of elements. The fact that the modern scientific theories you list are better than their precursors, does not mean their precursors were useless.
You have a false dichotomy going here. If you know of someone who “knows how human cognition works on all scales”, or even just a theory of cognition as powerful as Newton’s theory of mechanics is in its domain, then please, link! But if such a theory existed, we wouldn’t need to be having this discussion. A strong theory of cognition will descend from a series of lesser theories of cognition, of which control theory is one step.
Unless you have a better theory, or a convincing reason to claim that “no-theory” is better than control theory, you’re in the position of an elementalist arguing that phlogiston theory should be ignored because it can’t explain heat generated by friction—while ignoring the fact that while imperfect, phlogiston theory is strictly superior to elemental theory or “no-theory”.
You’ve misunderstood my emphasis. I’m an engineer—I don’t insist on correctness. In each case I’ve picked above, the emphasis is on a deeper understanding (a continuous quantity, not a binary variable), not on truth per se. (I mention correctness in the Coriolis example, but even there I have Newtonian mechanics in mind, so that usage was not particularly accurate.)
My key perspective can be found in the third paragraph of this comment.
I’m all for control theory as a basis for forming hypotheses and for Seth Roberts-style self-experimentation.
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
That’s kind of confusing.
What I said was:
One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
And you gave an argument that some correct things are useful. Bravo.
However, you did not dispute the part where “useful” almost always comes before “correct”… thereby demonstrating precisely the confusion I posted about.
Useful and correct are not the same, and optimizing for correctness does not necessarily optimize usefulness, nor vice versa. That which is useful can be made correct, but that which is merely correct may be profoundly non-useful.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
(To the extent you would pick the latter, you likely hold an irrational bias… which would also explain the fanboy outrage and downvotes that my comments on this subject usually provoke here.)
I did not simply argue that some correct things are useful. I pointed out that every example of usefulness you presented can be augmented beyond all recognition with a deeper understanding of what is actually going on.
Let me put it this way: when you write, “how were people were to start and put out fires for millennia...” the key word is “start”: being satisfied with a method that works but provides no deep understanding is stagnation.
Ever seeking more useful methods without seeking to understand what is actually going on makes you an expert at whatever level of abstraction you’re stuck on. Order-of-magnitude advancement comes by improving the abstraction.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
I would also pick the former, provided my number one choice was not practical (perhaps due to time or resource constraints). The number one choice is to devote time and effort to making the true theory practicable. But if you never seek a true theory, you will never face this choice.
ETA: I’ll address:
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
by saying that you are arguing against, and I am arguing for:
But things that are correct are usually rather useful, and things that are not correct are less so.
I agree with Cyan, but even more basically, the set of correct beliefs necessarily includes any and all useful beliefs, because anything that is useful but incorrect can be derived from correct beliefs as well (similar to Eliezer’s Bayesians vs. Barbarians argument).
So, probabilistically, we should note that P(Useful|Correct)>P(Useful|Incorrect) because the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs. More importantly, as Sideways notes, more correct beliefs produce more useful effects; we don’t know now whether we have a “correct” theory of genetics, but it’s quite a bit more useful than its predecessor.
You still don’t get it. Correct beliefs don’t spring full-grown from the forehead of Omega—they come from observations. And to get observations, you have to be doing something… most likely, something useful.
That’s why your math is wrong for observed history—humans nearly always get “useful” first, then “correct”.
Or to put it another way, in theory you can get to practice from theory, but in practice, you almost never do.
Let’s assume that what you say is true, that utility precedes accuracy (and I happen to believe this is the case).
That does not in any way change the math. Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
It doesn’t matter if you have an Einstein’s grasp of the physical laws, a Ford’s grasp of the mechanics, and a lawyer’s mastery of traffic law… you still have to practice in order to learn to drive.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
This is why, when one defines “rationality” in terms of strictly abstract mentations and theoretical truths, one tends to lose in the “real world” to people who have actually practiced winning.
And I wasn’t arguing that definition, nor did I perceive any of the above discussion to be related to it. I’m arguing the relative utility of correct and incorrect beliefs, and the way in which the actual procedure of testing a position is related to the expected usefulness of that position.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
To the extent that my argument and the one you mention here interact, I suppose I would say that “winning” should include not just individual instances, things we can practice explicitly, but success in areas with which we are unfamiliar. That, I suggest, is the role of theory and the pursuit of correct beliefs.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong, but it seems to me that robotics has gradually progressed from having lots of complicated theories and sophisticated machinery towards simple control systems and improved sensory perception… and that this progression happened because the theories didn’t work in practice.
So, AFAICT, the argument that “if you have a correct theory, things will go better” is itself one of those ideas that work better in theory than in practice, because usually the only way to get a correct theory is to go out and try stuff.
Hindsight bias tends to make us completely ignore the fact that most discoveries come about from essentially random ideas and tinkering. We don’t like the idea that it’s not our “intelligence” that’s responsible, and we can very easily say that, in hindsight, the robotics theories were wrong, and of course if they had the right theory, they wouldn’t have made those mistakes.
But this is delusion. In theory, you could have a correct theory before any practice, but in practice, you virtually never do. (And pointing to nuclear physics as a counterexample is like pointing to lottery winners as proof that you can win the lottery; in theory, you can win the lottery, but in practice, you don’t.)
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong
You are wrong. The above is a myth promoted by the Culture of Chaos and the popular media. Advanced modern robots use advanced modern theory—e.g. particle filters to integrate multiple sensory streams to localize the robot (a Bayesian method).
And this is even more true when considering elements in the formation of a robot that need to be handled before the AI: physics, metallurgy, engineering, computer hardware design, etc.
Without theory—good, workably-correct theory—the search space for innovations is just too large. The more correct the theory, the less space has to be searched for solution concepts. If you’re going to build a rocket, you sure as hell better understand Newton’s laws. But things will go much smoother if you also know some chemistry, some material science, and some computer science.
For a solid example of theory taking previous experimental data and massively narrowing the search space, see RAND’s first report on the feasibility of satellites here.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
Procedures are brittle. Theory lets you generalize procedures for new contexts, which you can then practice.
the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs.
I’m not sure I’d grant that unless you can show it mathematically. It seems to me there are infinite beliefs of all sorts, and I’m not sure how their orders compare.
Select an arbitrary true predicate sentence Rab. That sentence almost certainly (in the mathematical sense) is false if an arbitrary c is substituted for b. Thus, whatever the cardinality of the set of true sentences, for every true sentence we can construct infinitely many false sentences, where the opposite is not true. Thus, the cardinality of the set of true sentences is greater than the set of false sentences.
I don’t think that’s as rigorous as you’d like it to be. I don’t grant the “almost certainly false” step.
Take a predicate P which is false for Pab but true in all other cases. Then, you cannot perform the rest of the steps in your proof with P. Consider that there is also the predicate Q such that Qab is true about half the time for arbitrary a and b. How will you show that most situations are like your R?
I’m also not sure your proof really shows a difference in cardinality. Even if most predicates are like your R, there still might be infinitely many true sentences you can construct, even if they’re more likely to be false.
It’s definitely not rigorous, and I tried to highlight that by calling it a heuristic. Without omniscience, I can’t prove that the relations hold, but the evidence is uniformly supportive.
Can you name such a predicate other than the trivial “is not” (which is guaranteed for be true for all but one entity, as in A is not A) which is true for even a majority of entities? The best I can do is “is not describable by a message of under N bits,” but even then there are self-referential issues. If the majority of predicates were like your P and Q, then why would intelligence be interesting? “Correctness” would be the default state of a proposition and we’d only be eliminating a (relatively) small number of false hypotheses from our massive pool of true ones. Does that match either your experience or the more extensive treatment provided in Eliezer’s writings on AI?
If you grant my assertion that Rab is almost certainly false if c is substituted for b, then I think the cardinality proof does follow. Since we cannot put the true sentences in one-to-one correspondence with the false sentences, and by the assertion there are more false sentences, the latter must have a greater (infinite?) cardinality than the former, no?
You’re right. I was considering constructive statements, since the negation of an arbitrary false statement has infinitesimal informational value in search, but you’re clearly right when considering all statements.
If by “almost certainly false” you mean that say, 1 out of every 10,000 such sentences will be true, then no, that does not entail a higher order of infinity.
I meant, as in the math case, that the probability of selecting a true statement by choosing one at random out of the space of all possible statements is 0 (there are true statements, but as a literal infinitesimal).
It’s possible that both infinities are countable, as I am not sure how one would prove it either way, but that detail doesn’t really matter for the broader argument.
See the note by JGWeissman—this is only true when considering constructively true statements (those that carry non-negligible informational content, i.e. not the negation of an arbitrary false statement).
“Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”,
Which is it?
I think all the further you can go with this line of thought is to point out that lots of things are useful even if we don’t have a correct theory for how they work. We have other ways to guess that something might be useful and worth trying.
Having a correct theory is always nice, but I don’t see that our choice here is between having a correct theory or not having one.
Perhaps surprisingly, statistics has an answer, and that answer is no. If in your application the usefulness of a statistical model is equivalent to its predictive performance, then choose your model using cross-validation, which directly optimizes for predictive performance. When that gets too expensive, use the AIC, which is equivalent to cross-validation as the amount of data grows without bound. But if the true model is available, neither AIC nor cross-validation will pick it out of the set of models being considered as the amount of data grows without bound.
define: A theory’s “truthfulness” as how much probability mass it has after appropriate selection of prior and applications of Bayes’ theorem. It works as a good measure for a theory’s “usefulness” as long as resource limitations and psychological side effects aren’t important.
define: A theory’s “usefulness” as a function of resources needed to calculate its predictions to a certain degree of accuracy, the “truthfulness” of the theory itself, and side effects. Squinting at it, I get something roughly like:
usefulness(truthfulness, resources, side effects) = truthfulness * accuracy(resources) + messiness(side effects)
So I define “usefulness” as a function and “truthfulness” as its limiting value as side effects go to 0 and resources go to infinity.
Notice how I shaped the definition of “usefulness” to avoid mention of context specific utilities; I purposefully avoided making it domain specific or talking about what the theory is trying to predict. I did this to maintain generality.
(Note: For now I’m polishing over the issue of how to deal with abstracting over concrete hypotheses and integrating the properties of this abstraction with the definitions)
Your definition of usefulness fails to include the utility of the predictions made, which is the most important factor. A theory is useful if there is a chain of inference from it to a concrete application, and its degree of usefulness depends on the utility of that application, whether it could have been reached without using the theory, and the resources required to follow that chain of inference. Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
From pwno:
“Aren’t true theories defined by how useful they are in some application?”
My definition of “usefulness” was built with the express purpose of relating the truth of theories to how useful they are and is very much a context specific temporary definition (hence “define:”). If I had tried to deal with it directly I would have had something uselessly messy and incomplete, or I could have used a true but also uninformative expectation approach and hid all of the complexity. Instead, I experimented and tried to force the concepts to unify in some way. To do so I stretched the definition of usefulness pretty much to the breaking point and omitted any direct relation to utility functions. I found it a useful thought to think and hope you do as well even if you take issue with my use of the name “usefulness”.
Actions of high utility are useful. Of a set of available actions, the correct action to select is the most useful one. A correct statement is one expressing the truth, or probabilistically, an event of high probability. In this sense, a correct choice of action is one of which it is a correct statement to say that it is the most useful one.
It’s beside the point actually, since you haven’t shown that your info is either useful or correct.
What? If the odds of the lottery are uncertain, and your sample size is actually one, then surely it should shift your estimate of profitability.
Obviously a larger sample is better, and the degree to which it shifts your estimate will depend on your prior, but to suggest the evidence would be worthless in this instance seems odd.
It’s impossible for playing a lottery to be profitable, both before you ever played it, and after you won a million dollars. The tenth decimal place doesn’t really matter.
It’s impossible for playing a lottery to be profitable, both before you ever played it, and after you won a million dollars
I wonder what’s your definition of ‘profit’.
True story: when I was a child, I “invested” about 20 rubles in a slot machine. I won about 50 rubles that day and never played slot machines (or any lottery at all) again since then. So:
Expenses: 20 rubles.
Income: 50 rubles.
Profit: 30 rubles.
Assuming that we’re using a dictionary definition of the word ‘profit’, the entire ‘series of transactions’ with the slot machine was de-facto profitable for me.
It’s obvious that to interpret my words correctly (as not being obviously wrong), you need to consider only big (cumulative) profit. And again, even if you did win a million dollars, that still doesn’t count, only if you show that you were likely to win a million dollars (even if you didn’t).
The only way I can make sense of your comment is to assume that you’re defining the word lottery to mean a gamble with negative expected value. In that case, your claim is tautologically correct, but as far as I can tell, largely irrelevant to a situation such as this, where the point is that we don’t know the expected value of the gamble and are trying to discover it by looking at evidence of its returns.
That expected value is negative is a state of knowledge. We need careful studies to show whether a technique/medicine/etc is effective precisely because without such a study our state of knowledge shows that the expected value of the technique is negative. At the same time, we expect the new state of knowledge after the study to show that either the technique is useful, or that it’s not.
That’s one of the traps of woo: you often can’t efficiently demonstrate that it’s effective, and through intuition probably related to conservation of expected evidence you insist that if you don’t have a better method to show its effectiveness, the best available method should be enough, because it’s ridiculous to hold the claim to higher standard of proof on one side than on another. But you have to, the prior belief plays its part, the threshold to changing a decision may be too far away to cross by simple arguments. The intuitive thrust of the principle doesn’t carry over to expected utility because of the threshold, it may well be that you have a technique for which there is a potential test that could demonstrate that it’s effective, but the test is unavailable, and without performing the test the expected value of the technique remains negative.
I’m afraid I’m struggling to connect this to your originalobjections. Would you mind clarifying?
ETA: By way of attempting to clarify my issue with your objection, I think the lottery example differs from this situation in two important ways. AFAICT, the uselessness of evidence that a single person has won the lottery is a result of:
the fact that we usually know the odds of winning the lottery are very low, so evidence has little ability to shift our priors; and
that in addition to the evidence of the single winner, we also have evidence of incredibly many losers, so the sum of evidence does not favour a conclusion of profitability.
The analogy is this: using speculative self-help techniques corresponds to playing a lottery, in both cases you expect negative outcome, and in both cases making one more observation, even if it’s observation of success, even if you experience it personally, means very little for the estimation of expected outcome. There is no analogy in lottery for studies that support the efficacy of self-help techniques (or some medicine).
1) the range of conceivably effective self-help techniques is very large relative to the number of actually effective techniques
2) a technique that is negative-expected-value can look positive with small n
3) consequently, using small-n trials on lots of techniques is an inefficient way to look for effective ones, and is itself negative-expected-value, just like looking for the correct lottery number by playing the lottery.
In this analogy, it is the whole self-help space, not the one technique, that is like a lottery.
I don’t think the principle of charity generally extends so far as to make people reinterpret you when you don’t go to the trouble of phrasing your comments so they don’t sound obviously wrong.
If you see a claim that has one interpretation making it obviously wrong and another one sensible, and you expect a sensible claim, it’s a simple matter of robust communication to assume the sensible one and ignore the obviously wrong. It’s much more likely that the intended message behind the inapt textual transcription wasn’t the obviously wrong one, and the content of communication is that unvoiced thought, not the text used to communicate it.
it’s a simple matter of robust communication to assume the sensible one and ignore the obviously wrong.
But if the obvious interpretation of what you said was obviously wrong, then it’s your fault, not the reader’s, if you’re misunderstood.
the content of communication is that unvoiced thought, not the text used to communicate it.
All a reader can go by is the text used to communicate the thought. What we have on this site is text which responds to other text. I could just assume you said “Why yes, thoughtfulape, that’s a marvelous idea! You should do that nine times. Purple monkey dishwasher.” if I was expected to respond to things you didn’t say.
My point is that the prior under which you interpret the text is shaped by the expectations about the source of the text. If the text, taken alone, is seen as likely meaning something that you didn’t expect to be said, then the knowledge about what you expect to be said takes precedence over the knowledge of what a given piece of text could mean if taken out of context. Certainly, you can’t read minds without data, but the data is about minds, and that’s a significant factor in its interpretation.
If the text, taken alone, is seen as likely meaning something that you didn’t expect to be said, then the knowledge about what you expect to be said takes precedence
This is why people often can’t follow simple instructions for mental techniques—they do whatever they already believe is the right thing to do, not what the instructions actually say.
I don’t see how that’s relevant unless we already agree that this is like a lottery. My reading of conchis’s reply to your comment is that conchis doesn’t think we should have strong priors in that direction.
Why do you think this is a lottery-type situation?
This article is quite long. As general feedback, I won’t usually bother reading long articles unless they summarise their content up front with an abstract, or something similar. This post starts with more of a teaser. A synopsis at the end would be good as well: tell me three times.
FWIW, the original article on Kaj’s blog is formatted in a way that makes it much easier to read/skim than here.
I don’t mind the length; I second the “tell me three times”.
An observation: PJeby if you really have a self help product that does what it says on the tin for anyone who gives it a fair try, I would argue that the most efficient way of establishing credibility among the Less wrong community would be to convince a highly regarded poster of that fact. To that end I would suggest that offering your product to Eliezer Yudkowsky for free or even paying him to try it in the form of a donation to his singularity institute would be more effective than the back and forth that I see here. It should be possible to establish an mutually satisfactory set of criteria of what constitutes ‘really trying it’ beforehand to avoid subsequent accusations of bad faith.
What makes you think that that’s my goal?
Pjeby: If your goal isn’t to convince the less wrong community of the effectiveness of your methodology then I am truly puzzled as to why you post here. If convincing others is not your goal, then what is?
Helping others.
Do you expect anyone to benefit from your expertise if you can’t convince them you have it?
Either someone uses the information I give or they don’t. One does not have to be “convinced” of the correctness of something in order to test it.
But whether someone uses the information or not, what do I or my “expertise” have to do with it?
Someone is more likely to spend the time and effort to test something if they think it’s more likely to be correct.
It’s irrational of people who aren’t convinced that the information is useful to use it.
Either a tiger eats celery or it doesn’t. But the tiger has to be “convinced” that celery is tasty in order to taste it.
One of the most frustrating things about dealing with LW is the consistent confusion by certain parties between the terms “correct” and “useful”.
I said “one does not have to be convinced of the correctness of something in order to test it”, and you replied with something about usefulness. Therefore, there is nothing I can say about your response except that it’s utterly unrelated to what I said.
You are the one who introduced correctness into the argument. Alicorn said:
Feel free to read this as ‘convince them your expertise is “useful” ’ rather than your assumed ‘convince them your expertise is “correct” ’.
The underlying point is that there is a very large amount of apparently useless advice out there, and many self-help techniques seem initially useful but then stop being useful. (as you are well aware since your theory claims to explain why it happens)
The problem is to convince someone to try your advice, you have to convince them that the (probability of it being useful claimed benefit probability of claim being correct) is greater than the opportunity cost of the expected effort to try it. Due to others in the self-help market, the prior for it being useful is very low, and the prior for the claimed benefits equaling the actual benefits is low.
You also are running into the prior that if someone is trying to sell you something, they are probably exaggerating its claims to make a sale. Dishonest salespeople spoil the sales possibilities for all the honest ones.
If you can convince someone with a higher standing in the community than you to test your advice and comment on the results of their test, you can raise individual’s probability expectations about the usefulness (or correctness) of your advice, and hence help more people than you otherwise would have.
P.S. I did go to your site and get added to your mailing list. However, even if your techniques turn out positively for me, I don’t think I have any higher standing in this community than you do, so I doubt my results will hold much weight with this group.
Actually, I’m also running into a bias that merely because I have things to sell, I’m therefore trying to sell something in all places at all times… or that I’m always trying to “convince” people of something.
Indeed, the fact that you (and others) seem to think I need or even want to “convince” people of things is a symptom of this. Nobody goes around insisting that say, Yvain needs to get some high-status people to validate his ideas and “convince” the “community” to accept them!
If I had it all to do over again, I think I would have joined under a pseudonym and never let on I even had a business.
You are certainly right that “one does not have to be convinced of the correctness of something in order to test it.” But as you also said, immediately prior, “Either someone uses the information I give or they don’t.”
If we test information that we do not have reason to believe is useful, then we have a massive search space to cover. Much of the point of LW is to suggest useful regions for search, based on previous data.
So no, correctness is not a necessary condition of usefulness. But things that are correct are usually rather useful, and things that are not correct are less so. To the extent that you or your expertise are reliable indicators of the quality of your information, they help evaluate the probability of your information being useful, and hence the expected benefit of testing it.
Perhaps some parties on LW are actually confused by the distinction between truth and utility. I do not suspect Vladimir_Nesov is one of them.
Really? With what probability?
Or to put it another way: how were people were to start and put out fires for millennia before they had a correct theory of fire? Work metals without a correct atomic or molecular theory? Build catapults without a correct theory of gravity? Breed plants and animals without a correct theory of genetics?
In the entire history of humanity, “Useful” is negatively correlated with “Correct theory”… on a grand scale.
Sure, having a correct theory has some positive correlation with “useful”, but there’s usually a ton more information you need besides the correct theory to get to “useful”, and more often, the theory ends up being derived from something that’s already “useful” anyway.
That’s a shockingly poor argument. Who can constrain the future more effectively: someone who knows the thermodynamics of combustion engines, or someone who only knows how to start fires with a flint-and-steel and how to stop them with water? Someone who can use X-ray crystallography to assess their metallurgy, or someone who has to whack their product with a mallet to see if it’s brittle? Someone who can fire mortars over ranges requiring Coriolis corrections (i.e., someone with a correct theory of mechanics) or someone who only knows how to aim a catapult by trial and error? Someone who can insert and delete bacterial genes, or someone who doesn’t even know germ theory?
Someone who actually knows how human cognition works on all scales, or someone with the equivalent of a set of flint-and-steel level tools and a devotion to trial and error?
‘Correctness’ in theories is a scalar rather than a binary quality. Phlogiston theory is less correct (and less useful) than chemistry, but it’s more correct—and more useful!--than the theory of elements. The fact that the modern scientific theories you list are better than their precursors, does not mean their precursors were useless.
You have a false dichotomy going here. If you know of someone who “knows how human cognition works on all scales”, or even just a theory of cognition as powerful as Newton’s theory of mechanics is in its domain, then please, link! But if such a theory existed, we wouldn’t need to be having this discussion. A strong theory of cognition will descend from a series of lesser theories of cognition, of which control theory is one step.
Unless you have a better theory, or a convincing reason to claim that “no-theory” is better than control theory, you’re in the position of an elementalist arguing that phlogiston theory should be ignored because it can’t explain heat generated by friction—while ignoring the fact that while imperfect, phlogiston theory is strictly superior to elemental theory or “no-theory”.
You’ve misunderstood my emphasis. I’m an engineer—I don’t insist on correctness. In each case I’ve picked above, the emphasis is on a deeper understanding (a continuous quantity, not a binary variable), not on truth per se. (I mention correctness in the Coriolis example, but even there I have Newtonian mechanics in mind, so that usage was not particularly accurate.)
My key perspective can be found in the third paragraph of this comment.
I’m all for control theory as a basis for forming hypotheses and for Seth Roberts-style self-experimentation.
As best I can tell, you agree that what I said is true, but nonetheless dispute the conclusion… and you do so by providing evidence that supports my argument.
That’s kind of confusing.
What I said was:
And you gave an argument that some correct things are useful. Bravo.
However, you did not dispute the part where “useful” almost always comes before “correct”… thereby demonstrating precisely the confusion I posted about.
Useful and correct are not the same, and optimizing for correctness does not necessarily optimize usefulness, nor vice versa. That which is useful can be made correct, but that which is merely correct may be profoundly non-useful.
However, given a choice between a procedure which is useful to my goals (but whose “theory” is profoundly false), or a true theory which has not yet been reduced to practice, then, all else about these two pieces of information being equal, I’m probably going to pick the former—as would most rational beings.
(To the extent you would pick the latter, you likely hold an irrational bias… which would also explain the fanboy outrage and downvotes that my comments on this subject usually provoke here.)
I did not simply argue that some correct things are useful. I pointed out that every example of usefulness you presented can be augmented beyond all recognition with a deeper understanding of what is actually going on.
Let me put it this way: when you write, “how were people were to start and put out fires for millennia...” the key word is “start”: being satisfied with a method that works but provides no deep understanding is stagnation.
Ever seeking more useful methods without seeking to understand what is actually going on makes you an expert at whatever level of abstraction you’re stuck on. Order-of-magnitude advancement comes by improving the abstraction.
I would also pick the former, provided my number one choice was not practical (perhaps due to time or resource constraints). The number one choice is to devote time and effort to making the true theory practicable. But if you never seek a true theory, you will never face this choice.
ETA: I’ll address:
by saying that you are arguing against, and I am arguing for:
Deep theory has profound long-term impact, but is useless for simple stuff.
What is considered simple stuff is itself a function of that profound long-term impact.
I agree with Cyan, but even more basically, the set of correct beliefs necessarily includes any and all useful beliefs, because anything that is useful but incorrect can be derived from correct beliefs as well (similar to Eliezer’s Bayesians vs. Barbarians argument).
So, probabilistically, we should note that P(Useful|Correct)>P(Useful|Incorrect) because the space of correct beliefs is much smaller than the space of all beliefs, and in particular smaller than the space of incorrect beliefs. More importantly, as Sideways notes, more correct beliefs produce more useful effects; we don’t know now whether we have a “correct” theory of genetics, but it’s quite a bit more useful than its predecessor.
You still don’t get it. Correct beliefs don’t spring full-grown from the forehead of Omega—they come from observations. And to get observations, you have to be doing something… most likely, something useful.
That’s why your math is wrong for observed history—humans nearly always get “useful” first, then “correct”.
Or to put it another way, in theory you can get to practice from theory, but in practice, you almost never do.
Let’s assume that what you say is true, that utility precedes accuracy (and I happen to believe this is the case).
That does not in any way change the math. Perhaps you can give me some examples of (more) correct beliefs that are less useful than a related and corresponding (more) incorrect belief?
It doesn’t matter if you have an Einstein’s grasp of the physical laws, a Ford’s grasp of the mechanics, and a lawyer’s mastery of traffic law… you still have to practice in order to learn to drive.
Conversely, as long as you learn correct procedures, it doesn’t matter if you have a horrible or even ludicrously incorrect grasp of any of the theories involved.
This is why, when one defines “rationality” in terms of strictly abstract mentations and theoretical truths, one tends to lose in the “real world” to people who have actually practiced winning.
And I wasn’t arguing that definition, nor did I perceive any of the above discussion to be related to it. I’m arguing the relative utility of correct and incorrect beliefs, and the way in which the actual procedure of testing a position is related to the expected usefulness of that position.
To use your analogy, you and I certainly have to practice in order to learn to drive. If we’re building a robot to drive, though, it damn sure helps to have a ton of theory ready to use. Does this eliminate the need for testing? Of course not. But having a correct theory (to the necessary level of detail) means that testing can be done in months or years instead of decades.
To the extent that my argument and the one you mention here interact, I suppose I would say that “winning” should include not just individual instances, things we can practice explicitly, but success in areas with which we are unfamiliar. That, I suggest, is the role of theory and the pursuit of correct beliefs.
Actually, I suspect that this is not only wrong, but terribly wrong. I might be wrong, but it seems to me that robotics has gradually progressed from having lots of complicated theories and sophisticated machinery towards simple control systems and improved sensory perception… and that this progression happened because the theories didn’t work in practice.
So, AFAICT, the argument that “if you have a correct theory, things will go better” is itself one of those ideas that work better in theory than in practice, because usually the only way to get a correct theory is to go out and try stuff.
Hindsight bias tends to make us completely ignore the fact that most discoveries come about from essentially random ideas and tinkering. We don’t like the idea that it’s not our “intelligence” that’s responsible, and we can very easily say that, in hindsight, the robotics theories were wrong, and of course if they had the right theory, they wouldn’t have made those mistakes.
But this is delusion. In theory, you could have a correct theory before any practice, but in practice, you virtually never do. (And pointing to nuclear physics as a counterexample is like pointing to lottery winners as proof that you can win the lottery; in theory, you can win the lottery, but in practice, you don’t.)
You are wrong. The above is a myth promoted by the Culture of Chaos and the popular media. Advanced modern robots use advanced modern theory—e.g. particle filters to integrate multiple sensory streams to localize the robot (a Bayesian method).
And this is even more true when considering elements in the formation of a robot that need to be handled before the AI: physics, metallurgy, engineering, computer hardware design, etc.
Without theory—good, workably-correct theory—the search space for innovations is just too large. The more correct the theory, the less space has to be searched for solution concepts. If you’re going to build a rocket, you sure as hell better understand Newton’s laws. But things will go much smoother if you also know some chemistry, some material science, and some computer science.
For a solid example of theory taking previous experimental data and massively narrowing the search space, see RAND’s first report on the feasibility of satellites here.
IAWYC but
Procedures are brittle. Theory lets you generalize procedures for new contexts, which you can then practice.
I’m not sure I’d grant that unless you can show it mathematically. It seems to me there are infinite beliefs of all sorts, and I’m not sure how their orders compare.
A heuristic method that underlies my reasoning:
Select an arbitrary true predicate sentence Rab. That sentence almost certainly (in the mathematical sense) is false if an arbitrary c is substituted for b. Thus, whatever the cardinality of the set of true sentences, for every true sentence we can construct infinitely many false sentences, where the opposite is not true. Thus, the cardinality of the set of true sentences is greater than the set of false sentences.
I don’t think that’s as rigorous as you’d like it to be. I don’t grant the “almost certainly false” step.
Take a predicate P which is false for Pab but true in all other cases. Then, you cannot perform the rest of the steps in your proof with P. Consider that there is also the predicate Q such that Qab is true about half the time for arbitrary a and b. How will you show that most situations are like your R?
I’m also not sure your proof really shows a difference in cardinality. Even if most predicates are like your R, there still might be infinitely many true sentences you can construct, even if they’re more likely to be false.
It’s definitely not rigorous, and I tried to highlight that by calling it a heuristic. Without omniscience, I can’t prove that the relations hold, but the evidence is uniformly supportive.
Can you name such a predicate other than the trivial “is not” (which is guaranteed for be true for all but one entity, as in A is not A) which is true for even a majority of entities? The best I can do is “is not describable by a message of under N bits,” but even then there are self-referential issues. If the majority of predicates were like your P and Q, then why would intelligence be interesting? “Correctness” would be the default state of a proposition and we’d only be eliminating a (relatively) small number of false hypotheses from our massive pool of true ones. Does that match either your experience or the more extensive treatment provided in Eliezer’s writings on AI?
If you grant my assertion that Rab is almost certainly false if c is substituted for b, then I think the cardinality proof does follow. Since we cannot put the true sentences in one-to-one correspondence with the false sentences, and by the assertion there are more false sentences, the latter must have a greater (infinite?) cardinality than the former, no?
The cardinality of the sets of true and false statements is the same. The operation of negation is a bijection between them.
You’re right. I was considering constructive statements, since the negation of an arbitrary false statement has infinitesimal informational value in search, but you’re clearly right when considering all statements.
If by “almost certainly false” you mean that say, 1 out of every 10,000 such sentences will be true, then no, that does not entail a higher order of infinity.
I meant, as in the math case, that the probability of selecting a true statement by choosing one at random out of the space of all possible statements is 0 (there are true statements, but as a literal infinitesimal).
It’s possible that both infinities are countable, as I am not sure how one would prove it either way, but that detail doesn’t really matter for the broader argument.
See the note by JGWeissman—this is only true when considering constructively true statements (those that carry non-negligible informational content, i.e. not the negation of an arbitrary false statement).
Which is it?
I think all the further you can go with this line of thought is to point out that lots of things are useful even if we don’t have a correct theory for how they work. We have other ways to guess that something might be useful and worth trying.
Having a correct theory is always nice, but I don’t see that our choice here is between having a correct theory or not having one.
Both. Over the course of history:
Useful things → mostly not true theories.
True theory → usually useful, but mostly first preceded by useful w/untrue theory.
Aren’t true theories defined by how useful they are in some application?
Perhaps surprisingly, statistics has an answer, and that answer is no. If in your application the usefulness of a statistical model is equivalent to its predictive performance, then choose your model using cross-validation, which directly optimizes for predictive performance. When that gets too expensive, use the AIC, which is equivalent to cross-validation as the amount of data grows without bound. But if the true model is available, neither AIC nor cross-validation will pick it out of the set of models being considered as the amount of data grows without bound.
define: A theory’s “truthfulness” as how much probability mass it has after appropriate selection of prior and applications of Bayes’ theorem. It works as a good measure for a theory’s “usefulness” as long as resource limitations and psychological side effects aren’t important.
define: A theory’s “usefulness” as a function of resources needed to calculate its predictions to a certain degree of accuracy, the “truthfulness” of the theory itself, and side effects. Squinting at it, I get something roughly like: usefulness(truthfulness, resources, side effects) = truthfulness * accuracy(resources) + messiness(side effects)
So I define “usefulness” as a function and “truthfulness” as its limiting value as side effects go to 0 and resources go to infinity. Notice how I shaped the definition of “usefulness” to avoid mention of context specific utilities; I purposefully avoided making it domain specific or talking about what the theory is trying to predict. I did this to maintain generality.
(Note: For now I’m polishing over the issue of how to deal with abstracting over concrete hypotheses and integrating the properties of this abstraction with the definitions)
Your definition of usefulness fails to include the utility of the predictions made, which is the most important factor. A theory is useful if there is a chain of inference from it to a concrete application, and its degree of usefulness depends on the utility of that application, whether it could have been reached without using the theory, and the resources required to follow that chain of inference. Measuring usefulness requires entangling theories with applications and decisions, whereas truthfulness does not. Consequently, it’s incorrect to treat truthfulness as a special case of usefulness or vise versa.
Thank you—that’s an excellent summary.
From pwno: “Aren’t true theories defined by how useful they are in some application?”
My definition of “usefulness” was built with the express purpose of relating the truth of theories to how useful they are and is very much a context specific temporary definition (hence “define:”). If I had tried to deal with it directly I would have had something uselessly messy and incomplete, or I could have used a true but also uninformative expectation approach and hid all of the complexity. Instead, I experimented and tried to force the concepts to unify in some way. To do so I stretched the definition of usefulness pretty much to the breaking point and omitted any direct relation to utility functions. I found it a useful thought to think and hope you do as well even if you take issue with my use of the name “usefulness”.
Actions of high utility are useful. Of a set of available actions, the correct action to select is the most useful one. A correct statement is one expressing the truth, or probabilistically, an event of high probability. In this sense, a correct choice of action is one of which it is a correct statement to say that it is the most useful one.
It’s beside the point actually, since you haven’t shown that your info is either useful or correct.
FYI, The Others) is a group of fictional characters who inhabit the mysterious island in the American television series Lost.
pjeby will be more likely to notice this proposition if you post it as a reply to one of his comments, not one of mine.
Nope. The fact that you, personally, experience winning a lottery, doesn’t support a theory that playing a lottery is a profitable enterprise.
What? If the odds of the lottery are uncertain, and your sample size is actually one, then surely it should shift your estimate of profitability.
Obviously a larger sample is better, and the degree to which it shifts your estimate will depend on your prior, but to suggest the evidence would be worthless in this instance seems odd.
It’s impossible for playing a lottery to be profitable, both before you ever played it, and after you won a million dollars. The tenth decimal place doesn’t really matter.
I wonder what’s your definition of ‘profit’.
True story: when I was a child, I “invested” about 20 rubles in a slot machine. I won about 50 rubles that day and never played slot machines (or any lottery at all) again since then. So:
Expenses: 20 rubles.
Income: 50 rubles.
Profit: 30 rubles.
Assuming that we’re using a dictionary definition of the word ‘profit’, the entire ‘series of transactions’ with the slot machine was de-facto profitable for me.
It’s obvious that to interpret my words correctly (as not being obviously wrong), you need to consider only big (cumulative) profit. And again, even if you did win a million dollars, that still doesn’t count, only if you show that you were likely to win a million dollars (even if you didn’t).
The only way I can make sense of your comment is to assume that you’re defining the word lottery to mean a gamble with negative expected value. In that case, your claim is tautologically correct, but as far as I can tell, largely irrelevant to a situation such as this, where the point is that we don’t know the expected value of the gamble and are trying to discover it by looking at evidence of its returns.
That expected value is negative is a state of knowledge. We need careful studies to show whether a technique/medicine/etc is effective precisely because without such a study our state of knowledge shows that the expected value of the technique is negative. At the same time, we expect the new state of knowledge after the study to show that either the technique is useful, or that it’s not.
That’s one of the traps of woo: you often can’t efficiently demonstrate that it’s effective, and through intuition probably related to conservation of expected evidence you insist that if you don’t have a better method to show its effectiveness, the best available method should be enough, because it’s ridiculous to hold the claim to higher standard of proof on one side than on another. But you have to, the prior belief plays its part, the threshold to changing a decision may be too far away to cross by simple arguments. The intuitive thrust of the principle doesn’t carry over to expected utility because of the threshold, it may well be that you have a technique for which there is a potential test that could demonstrate that it’s effective, but the test is unavailable, and without performing the test the expected value of the technique remains negative.
I’m afraid I’m struggling to connect this to your original objections. Would you mind clarifying?
ETA: By way of attempting to clarify my issue with your objection, I think the lottery example differs from this situation in two important ways. AFAICT, the uselessness of evidence that a single person has won the lottery is a result of:
the fact that we usually know the odds of winning the lottery are very low, so evidence has little ability to shift our priors; and
that in addition to the evidence of the single winner, we also have evidence of incredibly many losers, so the sum of evidence does not favour a conclusion of profitability.
Neither of these seem to be applicable here.
The analogy is this: using speculative self-help techniques corresponds to playing a lottery, in both cases you expect negative outcome, and in both cases making one more observation, even if it’s observation of success, even if you experience it personally, means very little for the estimation of expected outcome. There is no analogy in lottery for studies that support the efficacy of self-help techniques (or some medicine).
It sounds like you’re saying:
1) the range of conceivably effective self-help techniques is very large relative to the number of actually effective techniques
2) a technique that is negative-expected-value can look positive with small n
3) consequently, using small-n trials on lots of techniques is an inefficient way to look for effective ones, and is itself negative-expected-value, just like looking for the correct lottery number by playing the lottery.
In this analogy, it is the whole self-help space, not the one technique, that is like a lottery.
Am I on the right track?
I don’t think the principle of charity generally extends so far as to make people reinterpret you when you don’t go to the trouble of phrasing your comments so they don’t sound obviously wrong.
If you see a claim that has one interpretation making it obviously wrong and another one sensible, and you expect a sensible claim, it’s a simple matter of robust communication to assume the sensible one and ignore the obviously wrong. It’s much more likely that the intended message behind the inapt textual transcription wasn’t the obviously wrong one, and the content of communication is that unvoiced thought, not the text used to communicate it.
But if the obvious interpretation of what you said was obviously wrong, then it’s your fault, not the reader’s, if you’re misunderstood.
All a reader can go by is the text used to communicate the thought. What we have on this site is text which responds to other text. I could just assume you said “Why yes, thoughtfulape, that’s a marvelous idea! You should do that nine times. Purple monkey dishwasher.” if I was expected to respond to things you didn’t say.
My point is that the prior under which you interpret the text is shaped by the expectations about the source of the text. If the text, taken alone, is seen as likely meaning something that you didn’t expect to be said, then the knowledge about what you expect to be said takes precedence over the knowledge of what a given piece of text could mean if taken out of context. Certainly, you can’t read minds without data, but the data is about minds, and that’s a significant factor in its interpretation.
This is why people often can’t follow simple instructions for mental techniques—they do whatever they already believe is the right thing to do, not what the instructions actually say.
That’s overconfidence, a bias, but so is underconfidence.
I don’t see how that’s relevant unless we already agree that this is like a lottery. My reading of conchis’s reply to your comment is that conchis doesn’t think we should have strong priors in that direction.
Why do you think this is a lottery-type situation?