I have taken the survey.
HungryHobo
Karma: 792
“could be a way to try to make sure that someone is friendly.”
I don’t believe in nested simulverse etc but I feel I should point out that even if some of those things were true waking up one way does not preclude waking up one or more of the other ways in addition to that.
Question 17 seems to lack an “other” category or at least an
Academics (on the research side)
Box.
No, I was comfortable with locking and the sort of stuff you’ll routinely see in high level languages where they lock away the complexity behind abstraction.
I’m talking about the “fun” of physical cores which few sane programmers touch.
Though most programmers outside of chip companies can’t touch those levels nowdays since even assembly is treated like a high-level language.
http://blog.erratasec.com/2015/03/x86-is-high-level-language.html#.VvQhiOaAnSg
I thought I was reasonably comfortable with basic parallel programming… then I spent a few hours talking with a very smart woman who specialized in low level parallel programming(close to the silicon) and talking about the kind of things that can go wrong.
It’s like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.
Which would be lovely if they actually claimed that this “life energy” was blood. They do not.
The first group of chiropractor has an evidence base and their limited claims actually work out in the real world and are very close to orthopedics.
The second is nowhere near the “church of science” and have little interest in it.
Science is not about your hypothesis sounding poetic.
Feedback from supervisors and feedback from reviewers is what the current system is mostly based on. We’re currently in a mostly-feedback system but it’s disorganised, poorly standardised feedback and the feedback tends to end a very short time after publication.
Some of the better journals operate blinded reviews so that in theory “anybody” should be able to publish a paper if the quality is good and that’s a good thing.
COMPare implies that preregistration didn’t solve all the problems but other studies have shown that it has massively improved matters.
Ok, look at chiropractors.
There are 2 schools of chiropractors. (ok, more but for this we can think of 2)
One who say “the body is reeeeaaallly complex but sometimes pushing and pulling the bones/muscles in the back around can help with back pain and sometimes can help with trapped nerves”
This is complex. It treats it as a complex problem where the intervention can help with a small subset of cases.
Then there’s the other group who have a really simple hypothesis that health problems are caused by disruptions to the flow of “life energy” to parts of the body and
Once the interference is gone, your life energy is able to flow to all parts of your body as it is needed.
and claiming that we need to
assure that our life energy is communicating with all the parts of the body, thus continuing to constantly create our bodies in a healthy manner.
They attribute almost all health problems to disruptions to the flow of energy.
This is really simple.
But simple(to say) does not mean correct.
Historically most people who attribute too many health problems to a single cause are quacks so if you want to be taken as a non-quack then you’re better off limiting your claims.
Having been through some of that process… it’s less than stellar.
That recent “creator” paper managed, somehow, to get through peer review and in the past I’ve been acutely aware that it’s been clear that sometimes reviewers have no clue about what they’ve been asked to review and just sort of wave it through with a few requests for spelling and grammar corrections.
To an extent it’s a very similar problem to ones faced in programming and engineering. Asking for more feedback is just the waterfall model applied to research.
To an extent, even if researchers weren’t being asked to publicly post their pre-reg getting them to actually work out what they’re planning to measure is a little like getting programmers to adopt Test Driven Development (write the tests, then write the code) which tends to produce higher quality output.
Despite that 8 years a lot of people still don’t really know what they’re doing in research and just sort of ape their supervisor. (who may have been in the same situation)
Since the system is still half-modeled on the old medieval master-journeyman-apprentice system you can also get massive massive massive variation in ability/competence so simply trusting in people being highly qualified isn’t very reliable.
The simplest way to illustrate the problem is to point to really really basic stats errors which make it into huge portions of the literature. Basic errors which have made it past supervisors, made it past reviewers, made it past editors. Made it past many people with PHD’s and not one picked up on them.
(This is just an example, there are many many other basic errors made constantly in research)
They’ve identified one direct, stark statistical error that is so widespread it appears in about half of all the published papers surveyed from the academic neuroscience research literature.
To understand the scale of this problem, first we have to understand the statistical error they’ve identified. This is slightly difficult, and it will take 400 words of pain. At the end, you will understand an important aspect of statistics better than half the professional university academics currently publishing in the field of neuroscience.
Let’s say you’re working on some nerve cells, measuring the frequency with which they fire. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice, and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.
When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have (in your imaginary experiment) this difference is statistically significant, which means it is unlikely to be due to chance. That’s a useful finding which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop in firing rate, but not as much – let’s say the drop is 15% – and this smaller drop doesn’t reach statistical significance.
But here is the catch. You can say that there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you cannot say that mutant cells and mormal cells respond to the chemical differently. To say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells.
Now, looking at the figures I’ve given you here (entirely made up, for our made up experiment) it’s very likely that this “difference in differences” would not be statistically significant, because the responses to the chemical only differ from each other by 15%, and we saw earlier that a drop of 15% on its own wasn’t enough to achieve statistical significance.
But in exactly this situation, academics in neuroscience papers are routinely claiming that they have found a difference in response, in every field imaginable, with all kinds of stimuli and interventions: comparing responses in younger versus older participants; in patients against normal volunteers; in one task against another; between different brain areas; and so on.
How often? Nieuwenhuis looked at 513 papers published in five prestigious neuroscience journals over two years. In half the 157 studies where this error could have been made, it was made.
It makes sense when you realize that many people simply ape their supervisors and the existing literature. When bad methods make it into a paper people copy those methods without ever considering whether they’re obviously incorrect.
If you don’t publicly pre-commit to what you’re going to measure then p-values become a bit meaningless since nobody can know if that was the only thing you measured.
If researchers are well organized then pre-reg should be almost free. On the other hand if they’re disorganized, winging it and making things up as they go along then pre-reg will look like a terrible burden since it forces them to decide what they’re actually going to do.
In theory yes but in practice it’s hard to untangle from papers where someone was a proper, real coauthor or contributor but died before publication.
Insightful, accurate and depressing.
On a related note:
Ben Goldacre along with colleagues has recently set up COMPare.
http://compare-trials.org/
For the last few years it’s been the norm that research in humans should be preregistered before the trial starts to avoid the file-drawer effect where negative trials don’t get published.
Without preregistration it’s hard to tell when someone has thrown a dart at a wall then built the dartboard around it.
It’s gradually been improving with a lot of research being preregistered but still published trials often don’t report what they said they were going to report or report things they didn’t preregister.
The dartboard is now there beforehand but people are still quietly building new dartboards around wherever the dart hits without mentioning it or mentioning that they were aiming at the original dartboard.
The COMPare project is doing something incredibly simple: Reading the paper. Reading the preregistered plan. Posting a public note on their website and sending a letter to the publishing journal pointing it out.
It’s embarrassing for journals because in theory they should have made sure that the papers matched what was preregistered during peer review.
They’ve had a range of responses, some journal editors like the ones at the BMJ have posted corrections while others have doubled-down like the editors at Annals of Internal Medecine, it’s really quite entertaining.
This is very interesting, particularly the person account of self-experimentation but may I suggest that if you want people to listen seriously it’s best to keep your claims minimalist.
“a possible palliative for some cases of Chronic Fatigue Syndrome” : excellent.
“and a panacea for most of the remaining unexplained diseases of the world.” : makes me immediately view the rest skeptically.
Attributing a large fraction of the worlds health ills to a single cause, even if only speculatively is a little bit of a red flag for crackpotism so if you want people to listen I’d suggest sticking to the smaller claims.
If some of the little claims pan out then try for some of the bigger ones.
To add an update from 2016.
Apparently some recent attempts to replicate, making sure to avoid the file drawer effect have not been able to replicate many of the older results.
https://osf.io/92dhr/wiki/home/
“The replication team ran that same experiment at 24 different labs, including ones that translated the letter e task into Dutch, German, French, and Indonesian. Just two of the research groups produced a significant, positive effect, says study co-author Michael Inzlicht of the University of Toronto. (One appeared to find a negative effect, a reverse-depletion.) Taken all together, the experiments showed no signs whatsoever of Baumeister and Tice’s original effect.”
.
“Meta-analyses are fucked,” Inzlicht warned me. If you analyze 200 lousy studies, you’ll get a lousy answer in the end. It’s garbage in, garbage out.
They’ve successfully trained related AI’s to play retro games, I believe including some with non-perfect information.
links to code etc in the youtube video description.
In the interests of making the math vaguely readable, any chance of giving the variables meaningful names?
I’m quite interested in how many of the methods employed in this AI can be applied to more general strategic problems.
From talking to a friend who did quite a bit of work in machine composition, he was of the opinion that tools for handling strategy tasks like go would also apply strongly to many design tasks like composing good music.
I have the feeling that a lot of the highest quality writers have gradually hived off to their own blogs and websites.
Lesswrong itself has sort of dropped bellow critical mass, interesting new updates are rare. There’s still high quality discussion but it’s often the same discussion much of the time.
To throw one out there, perhaps the first superintelligence was created by a people very concerned about AI risk and friendliness and one of it’s goals is simply to subtly suppress (by a very broad definition) unfriendly AI’s in the rest of the universe while minimizing disruption otherwise.
Same reason I don’t believe in god. As yet we have ~zero evidence for being in a simulation.
Your odds of waking up in the hands of someone extremely unfriendly is unchanged. You’re just making it more likely that one fork of yourself might wake up in friendly hands.