It’s interesting that you mention Rodney Brooks. I’ve always found his work poorly written and lacking in clarity despite being sympathetic to his views. He must come across better in person. As Shane points out though, Brooks’ work has the rare quality in AI that it is productive and has found widespread application in industry.
As for the Venture Capitalists, I don’t find it surprising that Silicon Valley VCs share some of your interests. It’s like discovering that software engineers share an interest in AD&D and collectibles. All these guys are enthusiastic about evolutionary psychology and cognitive science and such. I wonder if your perception of competence is a product of the “keyword search” approach to assessing other people that you frequently apply here; if they mention “evolution” and “probability” enough they get to be smart.
Most transhumanist ideas fall under the category of “not even wrong.” Drexler’s Nanosystems is ignored because it’s a work of “speculative engineering” that doesn’t address any of the questions a chemist would pose (i.e., regarding synthesis). It’s a non-event. It shows that you can make fancy molecular structures under certain computational models. SI is similar. What do you expect a scientist to say about SI? Sure, they can’t disprove the notion, but there’s nothing for them to discuss either. The transhumanist community has a tendency to argue for its positions along the lines of “you can’t prove this isn’t possible” which is completely uninteresting from a practical viewpoint.
If I was going to depack “you should get a PhD” I’d say the intention is along the lines of: you should attempt to tackle something tractable before you start speculating on Big Ideas. If you had a PhD, maybe you’d be more cautious. If you had a PhD, maybe you’d be able to step outside the incestuous milieu of pop sci musings you find yourself trapped in. There’s two things you get from a formal education: one is broad, you’re exposed to a variety of subject matter that you’re unlikely to encounter as an autodidact; the other is specific, you’re forced to focus on problems you’d likely dismiss as trivial as an autodidact. Both offer strong correctives to preconceptions.
As for why people are less likely to express the same concern when the topic is rationality; there’s a long tradition of disrespect for formal education when it comes to dispensing advice. Your discussions of rationality usually have the format of sage advice rather than scientific analysis. Nobody cares if Dr. Phil is a real doctor.