I completely agree with this comment. I don’t believe anyone is sufficiently epistemicly rational to have reached the threshold of actual competence, which is roughly 17 orders of magnitude more difficult that reaching the threshold of being known as awesome by your peer group. Thousands of men can work 12 hours a day for many decades without producing as much value as a single clever insight.
Friendly AI isn’t solved, the Singularity Institute has like 4 real researchers and none of them are really working on FAI even if some of them have seemingly clever ideas, some people like Mitchell Porter and Vladimir Nesov etc are working on Friendliness or very related problems but not many and it’s disorganized and no one thinks it’s important to actually address disagreements despite all this talk about how disturbed one should be by disagreement, Less Wrong is probably the most rational forum on the web and yet comments that are flat out wrong get upvoted too much, especially about tricky problems like FAI, et cetera.
We would not know if we were significantly below the necessary level of competence to have an important-in-hindsight insight. Hell, even the Singularity is just the opening of the rabbit hole. We could be missing some important things about the relevant philosophy. As a stupid example, the current common conception of the Singularity is “we fill the universe with utilitronium” which might not be nearly the correct framing in a Tegmark ’verse. Our comparative advantage is epistemic rationality, whether we like it or not. The reductionistic naturalistic cognitivist realist philosophy of Less Wrong is not satisfying even if it’s the best thing we have at the moment. I highly doubt that this is the point at which we can be satisfied with our epistemic ability and start moving lots of cognitive resources to building marginally rational communities. Following the leader doesn’t work without a smart enough leader, and there are no smart leaders (even if there are a few smart people).
I completely agree with this comment. I don’t believe anyone is sufficiently epistemicly rational to have reached the threshold of actual competence, which is roughly 17 orders of magnitude more difficult that reaching the threshold of being known as awesome by your peer group. Thousands of men can work 12 hours a day for many decades without producing as much value as a single clever insight.
Friendly AI isn’t solved, the Singularity Institute has like 4 real researchers and none of them are really working on FAI even if some of them have seemingly clever ideas, some people like Mitchell Porter and Vladimir Nesov etc are working on Friendliness or very related problems but not many and it’s disorganized and no one thinks it’s important to actually address disagreements despite all this talk about how disturbed one should be by disagreement, Less Wrong is probably the most rational forum on the web and yet comments that are flat out wrong get upvoted too much, especially about tricky problems like FAI, et cetera.
We would not know if we were significantly below the necessary level of competence to have an important-in-hindsight insight. Hell, even the Singularity is just the opening of the rabbit hole. We could be missing some important things about the relevant philosophy. As a stupid example, the current common conception of the Singularity is “we fill the universe with utilitronium” which might not be nearly the correct framing in a Tegmark ’verse. Our comparative advantage is epistemic rationality, whether we like it or not. The reductionistic naturalistic cognitivist realist philosophy of Less Wrong is not satisfying even if it’s the best thing we have at the moment. I highly doubt that this is the point at which we can be satisfied with our epistemic ability and start moving lots of cognitive resources to building marginally rational communities. Following the leader doesn’t work without a smart enough leader, and there are no smart leaders (even if there are a few smart people).