I think it stems from the Brennan’s World weirdtopia, and the idea that making knowledge freely available makes it feel worthless, while making it restricted to members of a secretive group makes it feel as valuable and powerful as it actually is.
If something is valuable and powerful, and (big if) it’s not harmful, plus it’s extremely cheap to reproduce I see no reason not to distribute it freely. My confusion was that Brennan’s world seems set in the future, and I got the sense that EY may have been in favor of it in some ways (perhaps that’s mistaken). Since it seemed to be set in the future of our world, I got the sense that the Singularity had already happened. Maybe I just need to get to the fun sequence, but that particular future really made me uneasy,
Perhaps it’s only powerful in the hands of the chosen few. If it’s in the open and it looks powerful, then other people try it and see less than amazing success, and it looks less and less cool until it stops growing. But by then it’s harder for the special few to recognize its value—or perhaps don’t want to associate themselves with it—and potential is wasted.
If instead the details are kept secret but the powers known publicly, then the masters of the craft are taken seriously and can suck up all the promising individuals.
I don’t know how he feels about it currently, but in the past he did endorse Brennan’s world as a better way to organize society post-Singularity. It started as a thought experiment about how to fix the problem that most people take science for granted and don’t understand how important and powerful it is, and grew into a utopia he found extremely compelling. (To the point where he specifically did not explain the rest of the details because it is too inefficient to risk diverting effort towards. This was probably an overreaction.) He talks about this in
The linked article ends with this; I think this part of context is necessary. Emphasis mine:
Right now, we’ve got the worst of both worlds. Science isn’t really free, because the courses are expensive and the textbooks are expensive. But the public thinks that anyone is allowed to know, so it must not be important. Ideally, you would want to arrange things the other way around.
As I understand it, the Conspiracy world is a mental experiment with different advantages and disadvantages. And a tool used to illustrate some other concepts in a storytelling format (because this is what humans pay more attention to), such as resisting social pressure, actually updating on a difficult topic, and a fictional evidence that by more rational thinking we could be more awesome.
But it’s not an optimal (according to Eliezer, as I understand the part I quoted) world. That would be a world where the science is open (and financially available, etc.) to everyone and yet, somehow, people respect it. (The question is, how to achieve that, given human psychology.)
I think it stems from the Brennan’s World weirdtopia, and the idea that making knowledge freely available makes it feel worthless, while making it restricted to members of a secretive group makes it feel as valuable and powerful as it actually is.
If something is valuable and powerful, and (big if) it’s not harmful, plus it’s extremely cheap to reproduce I see no reason not to distribute it freely. My confusion was that Brennan’s world seems set in the future, and I got the sense that EY may have been in favor of it in some ways (perhaps that’s mistaken). Since it seemed to be set in the future of our world, I got the sense that the Singularity had already happened. Maybe I just need to get to the fun sequence, but that particular future really made me uneasy,
Perhaps it’s only powerful in the hands of the chosen few. If it’s in the open and it looks powerful, then other people try it and see less than amazing success, and it looks less and less cool until it stops growing. But by then it’s harder for the special few to recognize its value—or perhaps don’t want to associate themselves with it—and potential is wasted.
If instead the details are kept secret but the powers known publicly, then the masters of the craft are taken seriously and can suck up all the promising individuals.
I don’t know how he feels about it currently, but in the past he did endorse Brennan’s world as a better way to organize society post-Singularity. It started as a thought experiment about how to fix the problem that most people take science for granted and don’t understand how important and powerful it is, and grew into a utopia he found extremely compelling. (To the point where he specifically did not explain the rest of the details because it is too inefficient to risk diverting effort towards. This was probably an overreaction.) He talks about this in
Eutopia is Scary
The linked article ends with this; I think this part of context is necessary. Emphasis mine:
As I understand it, the Conspiracy world is a mental experiment with different advantages and disadvantages. And a tool used to illustrate some other concepts in a storytelling format (because this is what humans pay more attention to), such as resisting social pressure, actually updating on a difficult topic, and a fictional evidence that by more rational thinking we could be more awesome.
But it’s not an optimal (according to Eliezer, as I understand the part I quoted) world. That would be a world where the science is open (and financially available, etc.) to everyone and yet, somehow, people respect it. (The question is, how to achieve that, given human psychology.)