How about testing our ideas?
Related to: Science: Do It Yourself, How To Fix Science, Rationality and Science posts from this sequence, Cargo Cult Science, “citizen science”
You think you have a good map, what you really have is a working hypothesis
You did some thought on human rationality, perhaps spurred by intuition or personal experience. Building it up you did your homework and stood on the shoulders of other people’s work giving proper weight to expert opinion. You write an article on LessWrong, it gets up voted, debated and perhaps accepted and promoted as part of a “sequence”. But now you’d like to do that thing that’s been nagging you since the start, you don’t want to be one of those insight junkies consuming fun plausible ideas forgetting to ever get around to testing them. Lets see how the predictions made by your model hold up! You dive into the literature in search of experiments that have conveniently already tested your idea.
It is possible there simply isn’t any such experimental material or that it is unavailable. Don’t get me wrong, if I had to bet on it I would say it is more likely there is at least something similar to what you need than not. I would also bet that some things we wish where done haven’t been so far and are unlikely to be for a long time. In the past I’ve wondered if we can in the future expect CFAR or LessWrong to do experimental work to test many of the hypotheses we’ve come up with based on fresh but unreliable insight, anecdotal evidence and long fragile chains of reasoning. This will not happen on its own.
With mention of CFAR, the mind jumps to them doing expensive experiments or posing long questionnaires with small samples of students and then publishing papers, like everyone else does. It is the respectable thing to do and it is something that may or may not be worth their effort. It seems doable. The idea of LWers getting into the habit of testing their ideas on human rationality beyond the anecdotal seems utterly impractical. Or is it?
That ordinary people can band together to rapidly produce new knowledge is anything but a trifle
How useful would it be if we had a site visited by thousands or tens of thousands solving forms or participating in experiments submitted by LessWrong posters or CFAR researchers? Something like this site. How useful would it be if we made such a data set publicly available? What if we could in addition to this data mine how people use apps or an online rationality class? At this point you might be asking yourself if building knowledge this way even possible in fields that takes years to study. A fair question, especially for tasks that require technical competence, the answer is yes.
I’m sure many at this point, have started wondering about what kinds of problems biased samples might create for us. It is important to keep in mind what kind of sample of people you get to participate in the experiment or fill out your form, since this influences how confident you are allowed to be about generalizations. Learning things about very specific kinds of people is useful too. Recall this is hardly a unique problem, you can’t really get away from it in the social sciences. WEIRD samples aren’t weird in academia. And I didn’t say the thousands and tens of thousands people would need to come from our own little corner of the internet, indeed they probably couldn’t. There are many approaches to getting them and making the sample as good as we can. Sites like yourmorals.org tried a variety of approaches we could learn from them. Even doing something like hiring people from Amazon Mechanical Turk can work out surprisingly well.
LessWrong Science: We do what we must because we can
The harder question is if the resulting data would be used at all. As we currently are? I don’t think so. There are many publicly available data sets and plenty of opportunities to mine data online, yet we see little if any original analysis based on them here. We either don’t have norms encouraging this or we don’t have enough people comfortable with statistics doing so. Problems like this aren’t immutable. The Neglected Virtue of Scholarship noticeably changed our community in a similarly profound way with positive results. Feeling that more is possible I think it is time for us to move in this direction.
Perhaps just creating a way to get the data will attract the right crowd, the quantified self people are not out of place here. Perhaps LessWrong should become less of a site and more of a blogosphere. I’m not sure how and I think for now the question is a distraction anyway. What clearly can be useful is to create a list of models and ideas we’ve already assimilated that haven’t been really tested or are based on research that still awaits replication. At the very least this will help us be ready to update if relevant future studies show up. But I think that identifying any low hanging fruit and design some experiments or attempts at replication, then going out there and try to perform them can get us so much more. If people have enough pull to get them done inside academia without community help great, if not we should seek alternatives.
- 's comment on Truth: It’s Not That Great by (5 May 2014 11:14 UTC; 59 points)
- 's comment on Experimental psychology on word confusion by (14 Sep 2012 6:42 UTC; 9 points)
- 's comment on Open Thread, September 15-30, 2012 by (18 Sep 2012 7:08 UTC; 7 points)
- 's comment on Dealing with trolling and the signal to noise ratio by (5 Nov 2012 21:14 UTC; 1 point)
I’m happy to see a push for increased empiricism and scientific effort on LW. But… I wish there were more focus on the word “how,” and less focus on the word ‘we.’
Three articles come to mind: To Lead You Must Stand Up, First, Try to Make it to the Mean, and Money: The Unit of Caring. (Only the first part of the second article will be directly relevant, but the latter parts are indirectly relevant.)
That is:
First, there’s insufficient focus on what concrete steps you are taking to move the culture in that direction. (Writing blog posts exhorting action does not count for much. Do you think The Neglected Virtue of Scholarship would have shifted community actions as much if lukeprog hadn’t followed it up by writing posts with massive reference lists?) The reference to yourmorals.org is fine, but what made that site important was a particular feature, not its goal or its structure. If you’ve thought of a similar feature that someone (ideally you) could code up, great! I will send as much karma as I can towards the person that makes that happen. But this is even more general than a call for better / easier rationality tests and exercises, and thus even less likely to cause concrete action.
Second, it really does help to be a specialist and know the prior art in a subject. The central lesson of experimental psychology is probably “designing experiments that test what you want them to test is really, really hard.” If there’s a specialist out there researching this stuff, then I would be happy to take part in any experiments they post on LW, and I suspect that many others here would be as well. If CFAR moves from advocacy and education to research (on cognitive science, not education), I again expect that I’d be willing to participate and so would others.
Similarly to trying to push the boundaries of life extension rather than simply making past the mean of the life expectancy, trying to push the boundaries of science when you don’t know where those boundaries are is fundamentally mistaken. Knowing what experiments have already been done and what they actually show should be a major input into what you test. The Neglected Virtue of Scholarship calls out Eliezer on exactly that- “er, your n=1 theory of procrastination seems to disagree with n>1 research.” I remember being fascinated by all the variants of the Wason selection task described in Thinking and Deciding. I had previously only been familiar with the basic one, and the implications of both the original and the variations are far stronger than the implications of just the original.
(Note that one of the strengths of LW might be that you gather a bunch of neurologically similar people, who can share with each other knowledge and experience not useful to the general population. I have the same experience of procrastination as Eliezer, and learning that someone else out there has that issue is valuable knowledge. Given general human neurodiversity, looking for things that help everyone is probably going to be less useful than narrowing your view.)
Third, why try to train citizen scientists when we could make better use of specialist scientists? Gary Drescher posted here, but hasn’t in over a year. What would make LW valuable enough to him for him to post here? XiXiDu managed to attract the attention of some experts in AI. What would make LW valuable enough to them for them to post here?
I agree with training citizen scientists in the sense of training empiricists (who will then naturally apply science to their lives). I think that LW having a culture of supporting science- both with dollars and volunteerism- would be better than not. But I don’t see you addressing the engineering problems with moving from one culture to the other, instead of just signalling that you would prefer the other culture.
A little over a week ago me and two other LWers started doing research on the possibilities of an online rationality class. The goal the project is to have an official proposal as well as a beta version ready in a few months. Besides this hopefully helping spread friendly memes and giving publicity, we aim to figure out if this can be used as a tool to make progress on the difficult problems of teaching and measuring rationality. Best way to figure it out is to try and use it that way as we iterate.
I name dropped the proposal in the OP but since we started so recently it felt odd writing an article about that first.
I kind of meant this under “attracting the right crowd” but I should have made it explicit.
The reason for this is that I’m unsure how to do this and didn’t want to get people locked into my plan to change it. Also I hoped “come up with stuff that needs testing!” would show me if I was wrong or not on the insufficient emphasis on empiricism in the community.
I was rereading my comments (because of this post) and noticed this:
Apparently, links to drafts of novel, interesting math papers.
It would very much surprise me if the goal of creating a space where experts in AI consider it valuable to post were not in tension with the goal of doing large-scale outreach (a la HPMOR).
This suggests that if LW’s sponsors want to attract participation from experts in AI who sufficiently embrace the community’s norms (e.g., that the rational position for anyone researching AI is to concentrate first on reliable Friendliness, lest they inadvertently destroy the world) to be valuable, it might be practical to do it in a space separate from where it attracts participation from the community at large (e.g., folks like me).
Agreed. My expectation of a first step would be something like inviting relevant experts to give a talk at CFAR. Possibly tape it / have someone at CFAR turn it into a post and put it on LW, but the immediate goal would be the relationship with the expert, which is probably much more easily done by having one visible and friendly person at CFAR they know.
Money?
If a guy with a webcomic can raise a million dollars to make a video game, maybe we can fund research by people who know how to do research well.
Huh, that’s not a bad idea, actually. Had there ever been an attempt by the SIAI or CFAR (I’m not sure which is responsible for what, to be honest) to Kickstarter some project related to rationality or AI ? Something with a clear, measurable goal, like, “give us a million dollars and we’ll give you a Friendly Oracle-grade AI”—though less ambitious than that, obviously.
If you can convince 12,000 people to donate an average of $100 to an efficient research organization, great! I am not optimistic about your prospects, though.
I’m curious as to hear an example or two of what sort of experiments you had in mind (and the models they’d be testing) when writing this article. A brief, none-too-thorough attempt on my own part kept hitting a couple of walls. I agree with your sentiment that simple surveys may fall prey to sampling biases, and I wonder how we would acquire the resources to conduct experiments methodologically sound enough that their results would be significant, the way CFAR is doing them, with randomized controlled trials and the like.
Luminosity is an example of a sequence that I liked but thought of as something that has many predictions that need testing . Methods of teaching rationality also seem like an obvious area for gain anyone who is interesting in that may be interested in a project I started to tackle this.
I implicitly argue that CFAR would be better of doing this kind of thing too. We are at the dawn of an era of computational social science, transitioning to a more data driven model seems promising. If that is the course they pick I see no reason to make a strong distinction between CFAR and the wider LessWrong community.
We can raise funds via kickstarter or just regular donations to cover expenses, as was done in the example of Citizen Genetics. Competitions like the quantified health prize could be used by CFAR to pick which of the community proposed projects it would like to happen and perhaps fund it. For things like processing time and bandwith you already have LWers willing to give it away for free.
We wouldn’t need that many resources. A website of the yourmorals.org kind (that is used for academic research) is something well within our reach (YourBrain, YourReasoning, RURational?). As to the cognitive resources, don’t we have enough Bayesians and people with basic programming knowledge that given resources like common crawl we could come up with interesting research? That we have many AI experts on the site only means we also have the know how to tackle big data.
It seems to me that most of the trepidation I associate with this problem comes from the difficulty of designing good experiments that are actually worth performing and would actually discriminate between two plausible hypotheses, not so much not knowing how to get enough research subjects.
I am not an experimental design specialist, but I am happy to offer whatever relevant advice I can. Incidentally, I am not far from the Bay Area in the next few months.
This is indeed a hard problem to tackle. I do outright say that we may need to attract a different crowd to get all of the skill set needed and suggest data mining as a form of empiricism we are already intellectually equipped to handle because of our demographics. It was my hope that we would start comping a list of things that would be neat to test and then hopefully identify some low hanging fruit.
I think wecan with not too many resources reach the stage where we are doing work as good as the kind we wouldn’t have second thoughts about citing as relatively strong evidence in articles.
I like the idea and agree this could be useful, but our first focus should be on reproducing well-established results to make sure that Less Wrong has the ability to so. Afterward, if we succeed, then it makes sense to proceed cautiously to replicating weak results or testing new hypotheses. Otherwise, if we don’t succeed in reproducing well-established results, then we lack the ability to investigate more speculative ideas.
We face numerous additional burdens over experimentalists in academia, two of which stand out to me: (1) a currently unknown sampling bias and response rate, and (2) a general inability to blind our Less Wrong subjects to the hypotheses and predictions of our experiments.
It may be instructive to take a look at some of the recent papers on online psychometric assessment in personality psychology. They have worked on establishing the effects of online assessment (as opposed to traditional paper-and-pencil or face-to-face assessments) and proposed guidelines for doing experiments online:
Buchanan, Tom. 2002. “Online Assessment: Desirable or Dangerous?” Professional Psychology: Research and Practice 33 (2): 148–154. doi:10.1037/0735-7028.33.2.148.
Davis, Robert N. 1999. “Web-based Administration of a Personality Questionnaire: Comparison with Traditional Methods.” Behavior Research Methods, Instruments, & Computers 31 (4) (December): 572–577. doi:10.3758/BF03200737.
Gosling, Samuel D., Simine Vazire, Sanjay Srivastava, and Oliver P. John. 2004. “Should We Trust Web-based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires.” American Psychologist 59 (2): 93–104. doi:10.1037/0003-066X.59.2.93.
Joubert, Tina, and Hendrik J. Kriek. 2009. “Psychometric Comparison of Paper-and-pencil and Online Personality Assessments in a Selection Setting.” SA Journal of Industrial Psychology 35 (1) (April 9): 78–88. doi:10.4102/sajip.v35i1.727.
Naus, Mary J., Laura M. Philipp, and Mekhala Samsi. 2009. “From Paper to Pixels: A Comparison of Paper and Computer Formats in Psychological Assessment.” Computers in Human Behavior 25 (1) (January): 1–7. doi:10.1016/j.chb.2008.05.012.
Ployhart, Robert E., Jeff A. Weekley, Brian C. Holtz, and Cary Kemp. 2003. “Web-based and Paper-and-pencil Testing of Applicants in a Proctored Setting: Are Personality, Biodata, and Situational Judgment Tests Comparable?” Personnel Psychology 56 (3) (September): 733–752. doi:10.1111/j.1744-6570.2003.tb00757.x.
Reips, Ulf-Dietrich. 2002. “Standards for Internet-based Experimenting.” Experimental Psychology 49 (4) (October 1): 243–256. doi:10.1026//1618-3169.49.4.243.
Rothstein, Mitchell G., and Richard D. Goffin. 2006. “The Use of Personality Measures in Personnel Selection: What Does Current Research Support?” Human Resource Management Review 16 (2) (June): 155–180. doi:10.1016/j.hrmr.2006.03.004.
Salgado, Jesús F., and Silvia Moscoso. 2003. “Internet-based Personality Testing: Equivalence of Measures and Assesses’ Perceptions and Reactions.” International Journal of Selection and Assessment 11 (2-3) (June): 194–205. doi:10.1111/1468-2389.00243.
Vecchione, Michele, Guido Alessandri, and Claudio Barbaranelli. 2012. “Paper-and-pencil and Web-based Testing: The Measurement Invariance of the Big Five Personality Tests in Applied Settings.” Assessment 19 (2) (June): 243–246. doi:10.1177/1073191111419091.
For the good of all of us, except the ones who are dead and haven’t opted for cryonics.
Also excepting the ones whose consciousness wasn’t uploaded into homicidal testing machines, of course.
Incredibly good idea! Why:
I was told by a professional developmental psychologist that it’s often the psychology assistants who have the creative ideas—the psychology training can get in the way. This may be due to problems like anchoring bias (where they’re anchoring to all that existing information from their education), bias blind spots (I’ve seen these caused by the ego that one can get due to having a high-level degree), and confirmation bias (which can only happen if you have preconceptions to confirm) or others.
Specifically because many LessWrong members are not as educated as the people you’d find writing a scientific journal, LessWrong could have some real gems. It occurs to me to think about Greek salons, fourchan /b and the U.S. virtue of freedom. Ancient Greek salons, from what I understand, were not structured like modern schools—basically the smart people got together and talked. But what do we know the ancient Greeks for? Having lots of ideas. Fourchan /b is lawless. What do we know them for? Creating a lot of internet memes. And why is freedom a virtue in the United States? Because among other things, freedom is credited with unleashing innovation.
LessWrong is much more free than a peer-reviewed journal because it does not enforce any educational requirement, it allows perspectives as diverse as Yvain’s to Will Newsome’s (sort of), and our ideas are much freer because we don’t all have that big “curse of knowledge” to contend with.
If we do this, could we discover that LessWrong is an innovation powerhouse?
Name three examples. Or even one to start. Using a template like:
Short and long description of the phenomenon being analyzed.
A review of the existing literature on the topic (including academic, of course).
A list of models which endeavor to explain it.
For each model, how it explains the phenomenon, what other known phenomena are covered, which of them are in agreement with the model, and which are in contradiction, and why.
A list of predictions each model makes, including the analysis of potential uncertainties.
Analysis of potential experiments which can check the predictions and possibly differentiate between models.
Review of the literature describing these or similar experiments done thus far.
When applicable (and it should be nearly always), suggested computer simulation of each potential experiment and expected outcomes (software is generally cheaper than wetware).
After that, the simulated data should be analyzed, models, their predictions and suggested experiments adjusted and the simulation repeated until it produces satisfactory results (what is satisfactory?). One can potentially discover that the proposed experiments have been done already in vivo and compare them with the results in simu.
Only after all this preliminary work is done, it makes sense to actually start doing live experiments.
In reality (= in academia), many of the steps above, especially the simulation, are skipped or short-circuited (experimental cognitive science is traditionally less precise than, say, physics), but there is no good reason they should be. As a bonus, any research done according to a template like that should have little trouble getting peer-reviewed and published.
Again, I would love to see at least one example.
It seems that you are suggesting turning LW into some sort of alternative circuit for scientific pubblication.
That seems an inefficient thing to attempt. If you do publishable research, publish it to a peer-reviewed academic journal or conference.
You might use LW as a place to exchange preliminary ideas and drafts. Personally I’m a bit skeptical that it would yield a significant increase of productivity: scientific research typically requires an high level of domain specialization, it’s unlikely that you would find many people with the relevant expertise on LW, and the discussion will be uninteresting for anybody else, but you might want to give it a try.
You can call it that. I call it refining the art of human rationality. I don’t think building new knowledge is something that magically only happens in a box designated Academia. Remember SI did years of research basically outside it, they only started publishing so they could attract more talent and as a general PR move, not because it was the most efficient way to do it. We are already an alternative circuit for scientific publication. This is exactly what we do every time we publish an article carrying some novel take on human rationality or some instrumentally useful advice. We are just bad at it.
You don’t seem to have read the related articles I cited. I strongly suggest you do.
I would also recommend you read Why Academic Papers Are A Terrible Discussion Forum. As to your invoaction of the somewhat broken formal peer review process that came into existence in the 20th century and is sadly still with us (I recommend you search Vladimir_M’s comment history for more information on arguments against it) and the even more broken journal system I don’t feel like attacking those particular applause lights right now.
Science is much more than its current flawed implementation. As I said if you can get this research, that we need to figure out if our original thinking and speculation is non-sense or not, done inside academia, great job! More power to you. But I do think we should be open to doing it ourselves when this is needed. The better ones clearly should also be posted on preprint sites like ArXiv. Indeed the very best work probably would be worth paying the price of the significant effort needed to craft papers respected journals are likely to accept.
Good enough so it counts as strong Bayesian evidence =/= Will be published by a peer reviewed journal
No, but Academia is optimized for that and has hundreds of years of demonstrated effectiveness and accumulated experience.
Is it perfect? No.
Can you build something better from ground up? I don’t think so, at least not at a cost smaller than the cost needed to improve it. Certainly the present LW doesn’t look remotely like a superior alternative.
And what did they accomplish? Pretty much nothing, AFAIK.
If that’s their main reason to publish then it seems that they are doing the right thing for the wrong reason.
Are they so deluded to seriously believe they will be the first to build an AGI? I hope not. Therefore, if they want to have a chance to influence the development of AI projects, the best they can do is to disseminate the results of their research in a way that will maximize the number of AI researchers who will notice it and take it seriously. And this way is not Less Wrong, or an Harry Potter fanfiction or meetups or minicamps or all the other stuff they do. It’s academic publishing. Academic publishing should be SI’s raison d’etre, not a PR move.
So far, AFAIK (I’m not a SI historian, so I might be mistaken) they published a few papers on philosophy, on the same sort of topics the FHI people do (some of these papers are co-authored with them, IIUC). I didn’t read all of them, but my impression is that they didn’t contribute particularly novel insights.
We have yet to see what results their current research on program equilibrium will yield.
I’ve skimmed them. The “Science: Do It Yourself” uses an examples that is flawed by a glaring methodological error from the start. That speaks lots of why scientific research, like many other activities, is something best left to professionals.
Actually the features of academia I’m criticizing are much younger than that. The modern peer review system is something Einstein didn’t have to deal with for example. If you think hundreds of years of scientific progress are a good track record for a system I have some news for you…
Obviously none of his great papers could have survived peer review. Some people argue that this was merely because of trivial stylistic issues, and could have been fixed by giving citations in correct format, and so on and so forth, so that they read like modern peer reviewed papers.
Perhaps
But the fact that he got his degree with a boring trivial paper, when he had several of his greatest papers in hand, suggests that there was no fixing them. If they could not be submitted for a degree, probably could never pass peer review. Peer review is in a sense a form of committee, and committees tend to be dumber than their dumbest member. The primary job of a committee, whatever its ostensible job, is to make sure that no one rocks the boat, just as the primary job of a bureaucracy, whatever its ostensible job, is to breed red tape with red tape to generate more red tape.
Getting a group of people to function together so that their output is smarter than any one of them is hard, a deep and unsolved problem. The normal outcome is that their output is dumber than any one of them.
The scientific community solved this problem from the late seventeenth century to late nineteenth or early twentieth century. Although engineering continues to advance, and more powerful tools such as DNA readers continue to advance science, science itself seemed to run out of puff after Einstein.
I’m guessing this post was down voted because of author not content because I can’t find anything wrong with the latter.
Yes this is evidence towards him not being sure those papers could be fixed.
Exactly, coordination is hard. Perverse incentives, Goodhart’s law, agency dilemma, etc.
See most non-profit organizations ever.
Stagnation in our time.
While I think you are right for most fields, I would argue we see a relatively healthy culture and even functional institutions when it comes mathematics since they have been making considerable progresses. I’m continually shocked at just how many of say recent advancements in evolutionary biology are basically rediscoveries of what Darwin himself said! In general reading and taking seriously the best of old thinkers is an excellent use of spare time for the intellectually curious.
Peter Thiel makes the case that outside of computers we aren’t seeing much advancement in engineering either since at least the 1970s. He cites cultural reasons but also notes we’ve effectively outlawed innovation in the world of “stuff” but not the world of “bits”, so those who like innovation go to Wall Street or Silicon Valley. Arguably the net impact of more people going to Wall Street to practice financial voodoo have been decidedly negative. If it was worth paying the opportunity costs for all those people to go to Sillicon Valley also may not be as clear cut as we may like to imagine, especially if you take Eliezer’s arguments about the dangers of AI seriously and remember that the area of “stuff” includes such fields such as energy and medicine which radically alter quality of life.
Not only do many confuse progress of technology for scientific progress, they are used to thinking about building up knowledge about the world and healthy institutions of science as being basically the same thing. Which isn’t true at all. We’ve had millennia of gains in naturalistic knowledge before we ever came up with the scientific method let alone the culture of science! Westerners from about 1700-1950 did something remarkably right to do so much with so little. What could they have done with the tools available today! I hope no one will bring up a low hanging fruit counterargument here, as it is hard to argue that what they picked hadn’t been low hanging fruit for a respectable and advanced civilizations like that of the Chinese as well.
No. I don’t know the author but downvoted the naive understanding of progress implied with “ran out of puff after Einstein”. It was cheap cynicism signalling that seemed misleading to me (especially since the earlier parts of the comment came across as authorative.)
Thank you for explaining this.
I also downvoted, and I actually considered not doing so because it was so far above the usual standards of what I’ve come to expect from sam0345, but I decided that if it were written by somebody else, I would have downvoted, for pretty much the same reasons wedrifid gave.
That doesn’t really follow. You could be missing a problem with the content, or someone else could mistakenly observe same. (Or both!) (Or there’s some third reason to downvote comments that is neither author nor content!)
I will edit that sentence since I see your point about it.
This is not something I’ve overlooked. I did say I didn’t find anything wrong with it, I expected to be corrected if wrong or change some minds if the down voters where wrong.
You have to admit that considering author in question my hypothesis is very likely.
Edit: Can down voters please explain why you’ve down voted this comment?
My guess differs from your own: Criticizing academia is as political, indeed more political, than criticizing women, blacks, gays, Islam, the poor, Mexicans, the underclass, the fatherless, and so on and so forth, but because academia does not proclaim itself as a victim group, but rather a victimizing group, no one can leap forth with outraged cries of racist sexist homobophobic Islamophobia. (“homophobic” intentionally misspelled)
Those who find my postings offensive are allowed to outraged on behalf of the poor victimized oppressed victims of victimization, but if outraged by unkind accounts of academia, have to find some new rationalization for outrage.
One may, of course, criticize academia for racism sexism etc, and be much loved for doing so, but criticizing academia for intellectual misconduct, accusing academia of ignorance, closed mindedness, dogmatism, and just plain not caring about the truth does not get one loved.
You seem to be under the impression that Einstein’s papers were not reviewed by professional physicists. That’s incorrect: They were reviewed by journal editors who were professional physicists.
The modern peer review system was invented because during the 20th century the submissions to journals greatly increased both in number and in sub-field specialization. While journals also increased in number and specialization, they couldn’t keep up with that and had to “outsource” the review process.
This is quite wrong.
Even Einstein’s field, theoretical physics, had significant progress until at least the mid-70s, when the Standard Model was completed. Subsequent stagnation was probably largely due to the difficulty of obtaining experimental data: testing all the features of the Standard Model required an enormous effort culminating in the LHC, and presently we can’t do experiments on Plank scale phenomena.
Other areas of science greatly progressed. Biology, for instance, is still far from stagnation.
But Einstein only needed one journal editor to decide that his paper was good stuff that would rock the boat, whereas under peer review, he would in practice need every peer reviewer to agree that his papers did not rock the boat.
Under the old system, he needed one of n to get published. Under the new system, it tends to be closer to n of n.
Consensus, as Galileo argued, produces bad science.
And, pretty obviously, we are getting bad science.
Recall the recent study reported in nature that only three of fifty results in cancer research were replicable.
The background to this replication study is that biomedical companies pick academic research to try to develop new medications—and they decided they needed to do some quality assurance.
The exact rules of peer review vary between different journals and conferences, but in general no single referee has veto power. If there is major disagreement between referees, they will discuss, and if they fail to form a consensus the journal editors / conference chairmen will step in and make the final decision, after possibly recruiting additional referees.
This seems to be a more accurate process than having a single editor making a decision based on only their own expertise.
That’s a false positive problem, while you seemed to be arguing that peer review generated too many false negatives.
Anyway, neither referees nor editors try to replicate experimental results while reviewing a paper. That’s not the goal of the review process.
The review process is not intended to be a scientific “truth” certification. It is intended to ensure that a paper is innovative, clearly written, easy to place in the context of the research in its field, doesn’t contain glaring methodological errors and is described in sufficient detail to allow experimental replication. Replication is something that is done by independent researchers after the paper is published.
The stagnation of theoretical physics in the last few decades is also due to information cascades and other sociological/political effects; see The Trouble with Physics by Lee Smolin.
There was no such stagnation. This is the period which saw M-theory, the holographic principle, and the twistor revival, three of the great theoretical advances of all time, and in general there was an enormous elevation in the level of technical knowledge. Smolin is just peeved that string theory is where all the action was.
The question is whether these theories correspond to reality.
Holography likely does, in some form, given that it pops up from every direction of research. The rest—who knows.
Are those on par with Einstein’s work? (Maybe they’re close enough—though I’d wait a bit longer before saying that; but if you count how many physicists are working today and how many were working in the early 20th century...)
They are not. Einstein took some existing math and the hints of new physics and built a beautiful model on top of it, with marvelous new predictions, all of which pan out. Then he did it again 10 years later.
Nothing like it has been repeated since. Creation of QM was close, but it was a collective effort over decades, still quite unfinished.
I understand M theory sufficiently well to be seriously underwhelmed.
M theory and the holographic principle suspiciously resemble postmodernism: insiders talking to each other in ways that supposedly demonstrate their erudition, without any external check to verify that they are actually erudite, or even understand each other, or even understand what they themselves are saying. Twistors are valid and erudite mathematics, but don’t seem to get us any closer to anything interesting.
M theory is just string theory only more so. The trouble with string theory as a theory of spacetime is that it takes place in a fixed space time background, thus inherently makes no sense whatever. If you start with a contradiction, you can deduce anything you please. The central problem in any quantum theory of spacetime is that you have no fixed spacetime to stand upon, and string theory just blithely ignores the problem. That is not an advance in theoretical physics, that is finding weak excuses to publish meaningless papers.
That’s just an approximation. Those situations (flat space, hyperbolic space) are really just asymptotically fixed—the form of the space-time in the infinite past or the infinite future is fixed. But in between, you can have topology change.
String theory in positively curved space may even allow for topologically distinct asymptotic outcomes, but that is still a topic of great confusion.
There is a standard paradigm for applying string theory to the real world—grand unification, broken supersymmetry, compactification. I’d give that about a 50% chance of being correct. Then there are increasingly unfamiliar scenarios, the extreme of which would be a theory in which you don’t even have strings or branes, but in which some of the abstract properties of string theory (e.g. the algebraic structure of the amplitudes) still hold. The twistors could swing either way here: twistorial variables may exist for an orthodox string scenario, but there may also be twistorial theories way outside the usual M-theoretic synthesis.
I don’t think string theory as it exists is capable of of describing a space time that undergoes topological change as a result of the dynamics of the strings. They talk about branes undergoing topological change, but they undergo topological change within a given background spacetime that acts without being acted upon.
And if it is capable of describing such an event, string theorists don’t really have any idea of how to make it do it.
People go into Quantum Gravity because it is the big unsolved problem, find they cannot solve it, but they have to publish papers anyway. And so they do, resulting in postmodern physics.
There is a string counterpart to the old idea of “spacetime foam”, it’s called a “Calabi-Yau crystal”. The crystal fluctuates and branes are defects in the crystal. There are more things in string theory, sam0345, than are dreamt of in your philosophy.
That is just people waving their hands fast to distract you from noticing that not only do you have no idea what they are saying, they have no idea what they are saying either: Much like postmodernism, hence I described it as “postmodern physics”
Now you’re being paranoid. This isn’t a bluff, these aren’t just words. Compactification on a Calabi-Yau is one of the basic ideas for how to get realistic physics out of string theory, and “crystal melting” is a model of its microscopic quantum geometry.
Yes, and there are so many Calabi-Yau manifolds that just knowing that the world is a ten-dimensional spacetime with six dimensions rolled up into some Calabi-Yau manifold yields hardly any falsifiable prediction at all.
Is there any way to tell whether a lack of major new science is the result of institutional problems or if it’s caused by an absence of major discoveries which could be made with current tools?
Never mind DNA itself, which was discovered post Einstein
What I’ve inferred from statements of key SI folk (most especially Luke) is that they don’t think this likely, but they think the possible futures in which it happens are vastly superior to the ones in which it doesn’t, so they’re working towards it anyway.
Yeah, this seems pretty plausible to me as well. (Though also pretty unlikely.)
FWIW, my understanding of SI’s original chosen strategy for making AI researchers take LW’s ideas about Friendliness seriously was to publicize the Sequences, which would improve the general rationality of people everywhere (aka “raise the sanity waterline”), which would improve the rationality of AI researchers (and those who fund them, etc), which would increase the chances of AI researchers embracing the importance of Friendliness, which would increase the chances of FAI being developed before UFAI, which would save the world.
From what I can tell, SI has since them moved on to other strategies for saving the world, like publishing the Sequences in book form, publishing popular fiction, holding minicamps, etc., but all built on the premise that “raising the sanity waterline” among the most easily reached people is a more viable approach than attempting to reach specific audiences like professional researchers.
That’s seems to be an inefficient approach.
Even if you accept the premise that you can “teach” rationality to AI researchers capable of building an AGI (who probably would not be idiots, but they might be indeed affected by biases), doing so it’s still an extremenly unfocused way to accomplish the task of advancing the state of the art on machine ethics.
If you want to advance the state of the art on machine ethics, then the most efficient way of doing it is to do actual research on machine ethics. If AI researchers don’t take machine ethics as seriously as you think they should, then the most efficient way to convince them is to put forward your arguments in forms and media accessible and salient to them.
Once you go for peer review, you may receive negative feedback, of course. That might mean two things: That your core claims are wrong, in which case you should recognize that, stop wasting your efforts and move to something else, or that your arguments are uncompelling or unclear, in which case you should improve them, since it is your responsibility to make yourself understood.
My admittedly incomplete understanding is that “raising the sanity waterline” activities have now been spun off to the Center for Applied Rationality, which is either planning to incorporate as a non-profit or already incorporated. This would then leave SIAI as focusing on the strictly AGI- and Friendliness-related stuff.
Ah. I’m aware of the SI/CFAR split, but haven’t paid much attention to what activities are owned by which entity, or how separate their staffs and resources actually are. E.g., I haven’t a clue which entity sponsors LW, if either, or even whether it’s possible to distinguish one condition from the other.
From the information available on their websites, it seems that LW is still operated by SI.
I suggest splitting it off an operating it as a charity separate from both SI and CFAR.
Actually judging clever articles by the rent they demonstrably pay in anticipated experience? This idea is too radical Konkvistador. Don’t you know that hand waving or reading papers is fun and testing is like … work?
A young and learning member calling reading papers “fun” without a second thought is already impressive progress when compared to the epistemic attitude of most people around us, I’d say.
LW posters have noticed many times that the most instrumentally rational people, hailed for making the world better or at any rate leaving a mark on it (Page & Brin, Warren Buffett, Linus Torvalds, maybe Thiel; among politicians either Gandhi, Churchill or Lee Kuan Yew—they wouldn’t have got along! - and maybe some older ones like Alexander II of Russia or the people behind the Meiji Restoration...), rarely behave like Eliezer or Traditional Rationality would want them to. They exploited some peculiar factors, innate or unintentionally acquired advantages (genes, lucky upbringing, broad life experience) that LW attempts to emulate through some written advice and group meetings. Most haven’t even heard of Bayes or can’t name a couple of fallacies! :)
At this stage, if an LW user actually uses the letter and spirit of LW materials to gain rent in some complicated, important area (like education, career, interpersonal relations, “Luminosity”, fighting akrasia) - well, that’s a pleasant surprise but an improbable prior. And some might not even pretend to heed the advice. E.g. my choice of education and career (Social Sciences) directly contradicts the common LW wisdom, that much of it is pure woo and will be made irrelevant in the transhuman world anyway. I can’t even formulate a “rationalist” argument against that wisdom, besides some vague guesses that principles of social organization and grand-scale value conflict like farmers vs. foragers—what LW likes to dismiss as “politics”—might stay important after we handle FAI, death or scarcity. For all the LW consensus knows, I might be insane for choosing to blow the next few years on empty talk instead of going the 80000 Hours way, or raising x-risk awareness by writing fiction, or something else “rationalist”.
Even our smallest real gains (openness, changing one’s mind, “luminosity”, intellectual rigor) are impressive, given just how ineffective or double-edged most deliberate attempts at instrumental rationality are. New Atheism, “pragmatic” politics (along the lines of moldbuggery), “PUA”, theology-based intellectual traditions like the Jewish ones—all claim to make you wiser, more truth-oriented, with better heuristics… yet all can have specific, awful, all-too-commonly seen negative effects on their real audiences.
+1 to that.
I also strongly agree with the fact that even the current small observable gains are impressive. I have observed an approximately 50% reduction in the average time it takes for me to explain foreign concepts for respective inferential distances, thanks to the reductionism and words sequences. While very anecdotal, if a 100% increase of efficiency in informal pedagogy is not impressive, please tell me where I can find all the other goldmines that so dramatically increase the ability to teach, and I’d also like an explanation why the hell it seems like no teachers anywhere are studying any of it.
Schools do not teach any critical thinking and for good reason. Ivan Illich wrote “Deschooling Society” in the 70s and John Taylor Gatto started writing the “The Underground History of American Education” in the 90s. Either should give you insight into why teachers do what they do, but Gattos’s “Weapons of Mass Instruction” is probably the best place to start. The short answer is that schools are designed from the top down to stunt the intellectual growth of children regardless of the intentions of teachers.
Maybe the result is that they stunt growth, but to infer intention from that is just an agency-fantasy. I would guess that the bereaucrats that actually think about the result have good intentions, even.
Eh, the Prussian school system was explicitly designed to create soldiers, and stunting intellectual growth is a part of that. It’s not much of a stretch to call it intentional.
I doubt that many school officials or politicians today know about the influences of the Prussian school system on e.g. the United States school system, or would guess that their present systems bear features deliberately designed to stunt intellectual growth.
I suspect that they mostly see the system that they were themselves educated in as normal by default, and only think to question the appropriateness of features that are specifically brought to their attention, and then only contemplate changing them in ways that are politically practical and advantageous from their positions. Expecting them to try and design and implement a school system that best meets their stated goals is like expecting a person to specify to a genie exactly how they want their mother removed from a burning building so as to save her life. The problem and its solution space simply doesn’t fall within the realms that they’re inclined to actually think about.
Indeed. (I should clarify that I was interpreting the original inventors of the Volksschule were the ‘top’, not, say, Arne Duncan.)
I do not know if you have read Gatto or not based on this. He points out that the system has no memory of its origin and that changes occur just like you describe with the result of deepening the problem. The last major school reform was GW Bush’s No Child Left Behind....if that tells you anything about who “fixes” the system.
No Child Left Behind was a stupid fix, but that doesn’t mean it was an ill intentioned fix.
I have actually just found the online text of “The Underground History of Education,” and started reading it, but so far am unimpressed and unlikely to finish it. I’m noticing a lot of cherrypicking to support his position, and he doesn’t give sources for his assertions at all (I went to the table of contents to look for a bibliography, and couldn’t find one, so I did a further search to see if this is the case in the print version and confirmed that the book contains no citations.)
I share his opinion that our current educational system is not well designed to get the best out of its students, but if I wanted to introduce someone to a writer who could effectively explain that point, I don’t think I’d recommend him. I’d probably recommend some of Eliezer’s essays, or maybe Paul Graham’s.
Oops, I accidentally only considered US and Canada. (tho I know little of what goes on in the American system, now that I think about it)
The US system took heavily from the Prussian school. The history is fascinating to say the least.
The statements of intent where made in writing and in speeches. I would do it for you but linking on the droid is not fun. Google “Rockefeller mencken quotes education” and the first link should lend some insight into the intent of the designers of the compulsory public school system. Gatto did a lot of research to support the thesis that schools are designed to dumb down the populace.
This may simply be jumping on an issue of semantics, but I’m concerned that this is really what he did, rather than doing a lot of research to find out whether the thesis was correct, or, more ideally, doing a lot of research before promoting the hypothesis to attention at all.
Well you can make wild speculations based off of my semantics or you can read for yourself. You seem to have chosen the former. Please return and clarify if you find his research faulty after you have read his work.
Well, the first link I get when I do the google search you suggested is this, and I’ve read that, but I’m not clear on what research of his you’re expecting me to read. The only work of his that appears to be available online is this, which contains assertions, but does not appear to support them with research as such.
It’s true that I haven’t read John Taylor Gatto’s work beyond that essay and the page you suggested reading yourself, but I was not wildly speculating based on semantics, I was making an educated guess considering that this is how people ordinarily behave. I assign a much higher prior to someone, say, hearing that the American school system is inspired by the Prussian system, which was largely concerned with producing good citizen-soldiers, and concluding that the American system must be deliberately designed to stymy creative thought, and looking for more evidence to back up that assertion, than I do to someone deciding to find out what the intentions behind the American school system are, doing extensive research, and concluding that its programs are actually purpose-designed to dumb down the populace.
What little work of his that I’ve found accessible online certainly doesn’t shift me away from that assessment.
Sounds scary. I’ll look into it and update as appropriate.
You are postulating quite the conspiracy tho. Much more likely it seems that a few b’crats went bonkers, the way you sometimes get UFO nuts out of the military.
Not really. To militaristic Prussia of the time, creating good soldiers was simply the same as creating good citizens, and was considered a worthy goal. No conspiracy required, just doing what seemed obviously correct at the time. And then the Prussian system was so ‘advanced’ and ‘modern’ and ‘successful’ that others copied it.
American experts did not all agree with the ‘military’ goal, but it was believed by the relevant experts that the same sorts of virtues applied to factory workers.
Now people try to actually educate children via this system. It’s like making minor tweaks to a torture device and wondering why it is ineffective at relieving headaches. You put some ibuprofen on the screws, tighten them some more, and subjects report slightly less intense headaches than last time.
Not creating effective soldiers puts you at a military disadvantage. If Prussia was a major power at the time, surely other countries feared them. If other countries felt it was necessary to stifle their populace in order to ensure that they were capable of defending themselves against Prussia (or to defend themselves against the countries that took after Prussia), perhaps stifling the populace was thought to be a “lesser evil”, a sacrifice they justified as part of an arms race.
Maybe this wasn’t an evil conspiracy, but a terrible consequence of the prospect of war.
What’s the bias for: “Ahh! We’re in mortal danger! Quick, everybody, become stupid!”
Bayesians vs. Barbarians
I wrote a quick introduction to Gatto’s claims of detrimental schooling practices that I think will give you a quick idea of whether it’s worth continuing to look into. Let me know what your reaction is? I’m curious.
What is the “good” reason? Or did you not mean to agree with this practice?
By ‘good’ reason I meant one consistent with the purpose or function of schooling. It is to be taken as having a touch of humor based on people’s misunderstanding of the function of school believing it to be synonymous with education.
Oh, okay. I guess I didn’t know your personality well enough yet to assume the correct things. Thanks. (:
I suspect that free public education is on average about the same everywhere. Guessing teacher’s password and rote memorization are the easiest ways to teach, and an average teacher is not very good at what she does, so this method shows up by default. The idea that the US education is built on the “Prussian school system [which] was explicitly designed to create soldiers” and that’s why it is so bad seems like a conspiracy theory.
I would like to know if there are examples to the contrary (i.e. countries where an average high-school graduate is adept at independent learning and critical thinking).
John Taylor Gatto won the New York State teacher of the year award in 1991 (New York state’s education website). His ambition to be a great teacher led him to the realization that the system itself is broken and he was so disgusted with it that he resigned. The claims that John Taylor Gatto makes are much worse than that they’re defaulting to the teacher’s password. You have no idea. Consider this: You obviously value rational thought. Learning about things like logical fallacies and biases is a no-brainer to you, right? Why are so many people learning them here, at LessWrong, for the first time? From what I know of American public schools, most of them don’t teach these. What could cause our school systems to teach us square dancing and rote memorization of thousands of spellings of words for the sake of polish, but leave out basic pieces required for rational thought? Ask yourself this:
If you were making the curriculum, and you knew the kids would be turned lose into the world complete with the right to vote at 18 would you find any excuse good enough to let them out with no familiarity of logical fallacies, biases, etc.?
If your answer to this is “no” you already know that something is wrong.
I have a radar for conspiracy theories too, but what he explains in The Seven Lesson School Teacher (in the first chapter of his book “Dumbing us Down”) got past my conspiracy theory radar and made it to “oh crap”. If you want to fast forward past the pretty obvious stuff, start at #3 in that link, and if you want to begin with “oh crap” start with #4.
I have no idea if his claim that the American school system was based on the Prussian school system in order to create obedient soldiers is correct, or whether seeing the effects of schooling as intentional is just a matter of seeing agency where there is none due to bias. However, the problems he describes are worth consideration. That, I’m sure of.
This was the only one of the education theorists that I studied while getting my teachning qualification that was remotely inspiring.
Man, Gatto spurred off so much thought for me. That was in my early 20′s so it’s not all readily coming to mind right now, but wow. I feel like… he explained so much. I’m not sure why you say he’s inspiring. So much of life that didn’t make any sense began to make sense after that. But that was one of the worst existential crises I’ve ever experienced. To realize that your whole life you had been stifled by the thing you thought was teaching you: abominable. There are horrors worse than death. That is one of them.
When I was 17, I decided to tear my whole reality apart because I noticed that it contained too many flaws. This was excruciating and terrifying. When I was 18, I had the undignified experience of realizing I could not allow myself to vote because I wasn’t taught to think critically and was still learning to. When I was in my early 20′s, I discovered logical fallacies and went “SOMEBODY WROTE THIS ALL DOWN!!?!!?? Why didn’t I know about this!?” I was a mess of a young woman—it took years of effort to put together a decently competent mind after all that.
Failing to teach reasoning skills in school is a crime against humanity.
This pattern matches to the standard failure mode where exceptional individuals assume that others are more like them and therefore more competent than they actually are. This causes them to conclude that institutions are more flawed than they actually are.
It’s at least partly right, but strikes me as weak evidence. Horace Mann and several of his contemporaries admired the Prussian system and introduced reforms to American schooling based on it—but he probably wasn’t trying to foster exactly the same features that later commentators have objected to. There have also been several major changes to American public schooling since then, many of them divergent with the development of the Prussian (and, later, German) school system.
It’s plausible to me that the American school system functionally prioritizes inculcating obedience to authority, whether it was designed that way or fell into that arrangement through a process of evolution. But that’s a claim that’s got to stand or fall on an analysis of the system as it currently exists, not of its remote origins.
My thesis is that to a good first approximation the purpose of schools, like that of the TSA, is to instill and ensure compliance in the populace, not to educate. I should probably go and read this Gatto, he’s come up before in discussions like this one.
I really question whether a “world full of leaders” would necessarily fail. These discussions about the original purpose of schooling seem to come down to that question. Is it that the leaders at the time didn’t want to give up leadership to have a strong populace, or is it that they had no clue how to organize a strong populace in a way that makes them as effective as stifling them does?
I mean, it’s pretty counter-intuitive that stifling a populace will make it more effective, even if it gives you the ability to organize them better.
And it’s really questionable that a populace full of leaders wouldn’t figure out how to organize itself.
On the one hand, we could argue that the information overload of a species that is rapidly gaining knowledge will worsen if their minds aren’t standardized somehow. We could also ask “Are there different ways to standardize, some stifling, some not?” and “Would standardizing them by stifling them cause more or less information overload compared with standardizing them around a theme of rational thought?”
The standardizing by stifling option reduces the number of new ideas being created, but prevents bad ideas from being culled, which allows them to build up.
The standardizing by rationality option doesn’t necessarily mean you need to stifle creativity (and I think creativity is necessary for rational thought—lest every decision you make be subject to the flaws inherent in an option set too small, like with false dichotomy) but it would cause people to cull a large number of ideas that they waste time on right now AND it would give them a way to agree on things. Right now, we have a bunch of people who believe in opinion. They say “Everyone has their opinion. Let’s respect each other’s opinion.” as if they cannot be proven true or false, more or less effective. I think the problem is they don’t know enough about how to test ideas.
Assuming that a populace made of soldiers makes a country safer may be incorrect, too. Why did people in Nazi Germany adopt the morals of the Nazis or fail to oppose them effectively? According to Dabrowski’s theory, there are 5 levels of of moral development, and the one that 75% of the population is at (level two) is characterized by it’s adoption of authority’s morals—they do not think for themselves about morality or realize their own hypocrisy or question their authority’s morality (that happens at level 3). They just follow it blindly.
I’ve heard people argue that we need schools like these to keep people organized and to have soldiers… but if the “organized” thinkers are going to result in a proliferation of useless ideas and the “soldiers” are liable to kill their own citizens as well as actual enemies, then we may be both more disorganized and less secure than if we were to choose some other school system.
Perhaps this is the key to the problem you pose—if the desired outcomes are organization and national security (as opposed to, say, wielding tyrannical power), then perhaps posing a better educational solution to the problems of organization and national security is the key that would change this.
I am very, very heartened to see that someone (Eliezer) has finally made progress in gathering people around a theme of refining rationality. That needed to happen. I’ve been thinking that needed to happen for years now—because the general population needs thinking skills, because gifted people are socially fractured, a million reasons.
I wonder if the people here have what it takes to invent an education system that is better at security and organization.
Another big thing that’s missing from school is the idea of applying one’s thinking to a significant decision, and then acting on it.
I call this problem “spending crucial developmental years in simulation land”.
You seem to be reading these comments out of context. Cliff notes: That is John Taylor Gatto’s claim—that the American school system is based on the Prussian school system which was designed to create soldiers and the Prussian belief was that this required them to do things that stunted intellectual growth. Shminux said JTG seemed like a conspiracy theorist and my comment was in response to that.
You should definitely read JTG. At least read “The Seven Lesson School Teacher” (linked in my previous comment) if nothing else.
True: I had only read a small part of the thread at that time. My intent was to point out that there are plenty of others here on LW who have considered questions like the one above, and who have (in my case independently) come to conclusions that align with those you attribute to JTG. (That’s not hugely surprising, given Eliezer’s educational background.)
I wasn’t aware of the Prussian connection specifically, Wikipedia seems to confirm that the Prussian system has inspired other countries (including mine). (Not that I trust WP overmuch, but other sources concur.)
The thing that’s wrong is that nobody knows about rationality, and of those who do, most don’t care.
Not everyone knows about heuristics and biases. A b’crat who did wouldn’t care enough to put his career on the line.
Would have upvoted just for this.
My sense is that the education system struggles with the transition between learning-to-read and reading-to-learn. Those of us who make that transition easily often struggle with teacher’s password issues, but teacher’s password is probably only the second most frequent failure mode in public education.
Somewhat off-topic: high schools anywhere don’t seem to explicitly teach the only essential skill a college student must need: learning to learn:
is not a question students learn to ask or answer. Everyone who completes a post-secondary education tends to come up with some sort of implicit heuristics that get them through, few do it consciously.
The International Baccalaureate explicitly acknowledges this as a key issue, and claims to teach these skills specifically.
I can speak from personal experience in saying that they utterly fail at this—the coursework alone might be slightly better, but the implementation is still much too reliant on teachers, and their behaviors has an effect several orders of magnitude greater than the standard material in how much the learning to learn is actually made part of the students versus just giving us a couple more passwords that seem to be about other passwords.
They also completely ignore the recursivity issue (learning to learn to learn to learn … ), which means students are left to deal with this problem on their own, lest they become stuck wasting enormous amounts of time attempting to perfectly optimize their learning methods and never accomplish any actual non-meta learning.
Mind, this is meant to be taken as a rant/anecdote. I might (though I wouldn’t say it’s highly likely) have just gotten the worst of it and the programme generally fares better, for all I know.
Thanks for actually being a little specific on that. I’ve heard it a lot, but my thot is always “but what does that even mean”.
It’s even worse than that. I don’t think anyone knows how to teach people to learn. Obviously, individuals have done it, but human knowledge does not appear to contain a process that basically any ordinary teacher can follow to cause any ordinary student to learn how to learn.
That’s one of the top ten things I think we’d have after a good Singularity that we don’t have now. Maybe even top five.
Perspective shift: Frequenting LW is a “peculiar factor”/acquired advantage.
I can give you plenty of rationalizations, if you need them. My mind is very good at coming up with rationalizations.
Ack, noticed some tribe blindness in myself here. Out of the examples you list in your last paragraph:
I can immediately think of negative effects each of these ideas have on their audiences, except for the first one, New Atheism. Of course (remarkable coincidence!) that happens to be the one that I have personal association with. Can you elaborate on the negative effects you were thinking of when you mentioned New Atheism?
The bad of New Atheism: Children playing with memetic weapons, with the safety off.
It’s good at diagnosing problems with existing institutions, bad at understanding people’s non-rational but deeply-rooted needs that are currently satisfied by religion.
It has failed to create even remotely plausible replacement institutions, due to this lack of understanding.
It is fundamentally parochial: it originated from, and continues to narrowmindedly focus on, the internal life of the intellectual 1%, with little understanding or interest in the other 99% of the bell curve.
Lack of patience, overconfidence, more about signalling intelligence than about persuading religious people, lack of empathy. Those are the problems that came immediately to mind when I thought about it. That’s not to criticize all of New Atheism, though. I think I like the basic idea of it.
Yes, those all make sense, thank you.
And I am also a fan of at least a good subset of each of the other three examples. It’s just good, as you say, to remember how fraught with nasty side effects this whole self-improvement thing can often be.
Trying to rationalize something like this is much worse in the long run. Intentionally acting irrationally is much better than acting the same way, but believing it to be rational.
This advice is more useful on the meta-level of having a community norm of accepting people who do this. (Do we already have such a norm? I know that I act as if such a norm exists, but I am unsure if others do.) The good thing about Less Wrongers is that you can get them to adopt such a norm just by explaining why it would be useful.
Possible use for this new thing: Seeing as we have had much recent discussion about what behaviour is, and is not, creepy, we could create a long list of potential behaviour, and study which people think it’s creepy or not. And if it makes a difference if you ask first. (Question 492: So is it okay if I ask you out on a Wednesday… while wearing a tutu, and there are exactly 3.9 people in the room?)
Once we had this useful dataset, we could evaluate potential rules for social interaction (“okay, under your plan, 42% of women under 25 say they’ll hate you, but with my scheme, 50% say they’ll like you!”) etc.
Yes meta data and analysis on how to improve the site and community is low hanging fruit when it comes to research projects.
A recent example of competent small scale self-experimentation.
In my case, I’m comfortable with statistics but don’t know where to find the data for questions that interest me. The fact that much research is nearly inaccessible if you’re not affiliated with a university or other large institution is also a problem.
I most certainly don’t think that. I’m not so sure many people on LW think that. The part of your map that talks about others’ maps looks suspect.
Intellectual speculation isn’t bad in itself. Actually, it’s fun! It’s only bad when people don’t know that they’re in fact merely speculating. I have nothing against people of LessWrong banding together to do experimental science but that wouldn’t mean LessWrong as a whole was progressing on the path towards greater virtue. It would only mean that some users decided to do something more than talking about stuff on a forum and other users, while certainly cheering for them, decided that this whole citizen science business is too much trouble for too little reward.
Perhaps, but in practice I see appeals to the sequences or a well liked article taken as very strong evidence for a position and almost never is the central point challenged.
I certainly wasn’t expecting everyone on LW to head in a more empirical direction in practice, that would be silly, since even filling out a form takes some time. I was hoping LW could become a place where projects like this do happen and are encouraged. Much like currently careful high quality scholarship does happen and is encouraged. That’s the reason I cited the positive effect of lukeprog’s article as an example of the kind of a sufficient magnitude of change.
Do you count cases where someone puts a link to an article in their argument as instances of that phenomenon? In such cases it might be hard to tell whether someone is being dogmatic or just providing a link to an already written elaboration of the ideas involved.
Getting it done:
First we could pre-test them in an inexpensive way for the purpose of figuring out which ones are worth the money for independent research. Then, because LessWrong gets several million hits a year, an appeal to donate could be placed on LessWrong pages asking for donations to pay for high-quality research from an organization with credibility.
Throw away rotten food, don’t try to make it into an omelette.
Waste not, want not! I don’t think this community is quite ready for the compost heap.
Some parts are, some parts aren’t, and it is worth discerning the bad from the good still. If your post reaches some of the good then I suppose it was indeed time decently spent.