Because making money is nontrivial and requires more than just intelligence?
Could I extrapolate your statement and conclude that what makes an AGI dangerous is not its intelligence, because that wouldn’t be sufficient? Or would you qualify your statement in that case?
Humans have lots of bugs in their brains, like difficulty getting themselves to work, fear of embarrassment, vulnerability to discouragement, difficulty acting on abstract ideas, etc. Good entrepreneurs have to overcome those bugs. An AGI wouldn’t have them in the first place.
It helps, and that’s why successful entrepreneurs are often pretty smart. If you’re a smart person who’s good at self-improvement, you can improve yourself Benjamin Franklin style (reading lots of business book summaries, trying to brainstorm how you can be more effective every evening, etc.), fix some brain bugs, and potentially make lots of money. My impression of successful entrepreneurs is that they are often self-improvement enthusiasts.
On the other hand, consider the “geek” versus “suit” stereotype. The “suit” is more determined and confident, but less intelligent. So it’s not clear that intelligence is correlated with possessing fewer of these bugs in practice. I’m not sure why this is, although I have a few guesses.
What do you mean by nontrivial (time-consuming?), and what more does it require than inteligence, and time? (why is trading your time for direct work on projects better than trading it for acquiring enough money to hire more people who would in the future have done together with you more work than you would have done by working alone this period of time?) How would you know how much luck is involved in the different ways of making money? Here’s hoping my questions aren’t really stupid, if they are do tell ^^
Being a good entrepreneur requires skill at transforming abstract ideas into action, self-promotion skills, domain knowledge in the industry you start your business, willingness to take risks, emotional stability, inclination for hard self-directed work in the face of discouraging criticism, intuition for how the economy works, sales skill, negotiation skill, planning skill, scrappiness, comfort with failure, etc. Most of this stuff is not required for researchers. And yes, it takes lots of time too.
In any case, SI already has lots of supporters who are trying to make money by starting businesses. In fact, their old president Michael Vassar recently left to start a company. The people working at SI are pretty much those who decided they were better fit for research/outreach/etc. than entrepreneurship.
Being a good entrepreneur requires skill at transforming abstract ideas into action, self-promotion skills, domain knowledge in the industry you start your business, willingness to take risks, emotional stability, inclination for hard self-directed work in the face of discouraging criticism, intuition for how the economy works, sales skill, negotiation skill, planning skill, scrappiness, etc. Most of this stuff is not required for researchers.
Saving the world doesn’t require any of those qualities?
Depends how you’re planning to save it. If your plan involves you writing brilliant papers, maybe not.
SI has some folks with the qualities I described, like Louie, who sold his web-based business several years ago. They also have a number of entrepreneurs on the board. And as you suspect, these entrepreneurs’ skills are useful for SI’s mission to save the world. But they’re not so useful that SI wants everyone with those skills to join them as an employee—they have limited funds.
SI does think about how to best allocate the human capital of people concerned with UFAI. But if you have a thoughtful suggestion for how they could allocate their human capital even better, I’m sure they’d love to hear it.
What puzzles me is why there hasn’t been an attempt to get a lot of rationally thinking people together and work on solving the problems of taking luck into acount, building a network of people in needed positions, speeding up the process...?
If you’ve got some brilliant idea, why don’t you implement it? Complaining that someone else should do it could makes things worse:
Humans tend to be especially interested in implementing ideas they have themselves. If you tell someone else about your idea, there’s no chance of them having independently and getting excited about working on it. If you’re not actually going to do anything, you might want to just share the groundwork for the idea without mentioning the idea itself, or deliberately describe the idea in crippled form. That way, someone else can come along, have the idea, and get inspired to work on it.
I’m not complaining.
Does ‘getting together as a group of intelligent, rationality embracing humans, and brainstorming ideas with shared powers’ count as such an idea as what you are talking about?
In any case, your idea sounds great to me. There are already attempts to do this in informal conversations, and through the existential risk career network:
But I’m sure we can do much better! In particular, the existential risks career network isn’t terribly active and could probably be improved. If you have suggestions, you could work with FrankAdamek; it’s his brainchild.
Because making money is nontrivial and requires more than just intelligence?
Could I extrapolate your statement and conclude that what makes an AGI dangerous is not its intelligence, because that wouldn’t be sufficient? Or would you qualify your statement in that case?
Humans have lots of bugs in their brains, like difficulty getting themselves to work, fear of embarrassment, vulnerability to discouragement, difficulty acting on abstract ideas, etc. Good entrepreneurs have to overcome those bugs. An AGI wouldn’t have them in the first place.
That’s a sizable assertion. There’s an important difference between “known to have no bugs” and “has no known bugs.”
It seems unlikely that an AGI would suffer from the same evolution inspired troubles that humans do. Might have some other bugs.
But surely intelligence is what enables humans to overcome the bugs in their brains?
It helps, and that’s why successful entrepreneurs are often pretty smart. If you’re a smart person who’s good at self-improvement, you can improve yourself Benjamin Franklin style (reading lots of business book summaries, trying to brainstorm how you can be more effective every evening, etc.), fix some brain bugs, and potentially make lots of money. My impression of successful entrepreneurs is that they are often self-improvement enthusiasts.
On the other hand, consider the “geek” versus “suit” stereotype. The “suit” is more determined and confident, but less intelligent. So it’s not clear that intelligence is correlated with possessing fewer of these bugs in practice. I’m not sure why this is, although I have a few guesses.
What do you mean by nontrivial (time-consuming?), and what more does it require than inteligence, and time? (why is trading your time for direct work on projects better than trading it for acquiring enough money to hire more people who would in the future have done together with you more work than you would have done by working alone this period of time?) How would you know how much luck is involved in the different ways of making money? Here’s hoping my questions aren’t really stupid, if they are do tell ^^
Being a good entrepreneur requires skill at transforming abstract ideas into action, self-promotion skills, domain knowledge in the industry you start your business, willingness to take risks, emotional stability, inclination for hard self-directed work in the face of discouraging criticism, intuition for how the economy works, sales skill, negotiation skill, planning skill, scrappiness, comfort with failure, etc. Most of this stuff is not required for researchers. And yes, it takes lots of time too.
In any case, SI already has lots of supporters who are trying to make money by starting businesses. In fact, their old president Michael Vassar recently left to start a company. The people working at SI are pretty much those who decided they were better fit for research/outreach/etc. than entrepreneurship.
Thanks, this makes sense!
Saving the world doesn’t require any of those qualities?
Depends how you’re planning to save it. If your plan involves you writing brilliant papers, maybe not.
SI has some folks with the qualities I described, like Louie, who sold his web-based business several years ago. They also have a number of entrepreneurs on the board. And as you suspect, these entrepreneurs’ skills are useful for SI’s mission to save the world. But they’re not so useful that SI wants everyone with those skills to join them as an employee—they have limited funds.
SI does think about how to best allocate the human capital of people concerned with UFAI. But if you have a thoughtful suggestion for how they could allocate their human capital even better, I’m sure they’d love to hear it.
Luck, networking/who you know, and time are all very, very important.
What puzzles me is why there hasn’t been an attempt to get a lot of rationally thinking people together and work on solving the problems of taking luck into acount, building a network of people in needed positions, speeding up the process...?
If you’ve got some brilliant idea, why don’t you implement it? Complaining that someone else should do it could makes things worse:
Humans tend to be especially interested in implementing ideas they have themselves. If you tell someone else about your idea, there’s no chance of them having independently and getting excited about working on it. If you’re not actually going to do anything, you might want to just share the groundwork for the idea without mentioning the idea itself, or deliberately describe the idea in crippled form. That way, someone else can come along, have the idea, and get inspired to work on it.
I’m not complaining. Does ‘getting together as a group of intelligent, rationality embracing humans, and brainstorming ideas with shared powers’ count as such an idea as what you are talking about?
I’m not sure exactly what you’re asking.
In any case, your idea sounds great to me. There are already attempts to do this in informal conversations, and through the existential risk career network:
http://www.xrisknetwork.com/
But I’m sure we can do much better! In particular, the existential risks career network isn’t terribly active and could probably be improved. If you have suggestions, you could work with FrankAdamek; it’s his brainchild.