Thanks Gears. Yeah, I’m quite confident that we cannot rely on scaling being a mandatory part of advances in order to monitor the AGI landscape. I believe dangerously potent advances can come from small research groups working with garage-hobbyist compute. I also think that scaling without algorithmic innovation can also lead to dangerously potent advances. And some combination of the two can also work. There are many paths to AGI, and we can’t effectively ward against it by guarding just the most expensive and obvious paths while ignoring the darker quieter paths out of the limelight.
I believe dangerously potent advances can come from small research groups working with garage-hobbyist compute.
Can I get some insights on this? My feeling is that while the human brain runs on very little energy, so does a LLM in operation, right now. The training of an LLM is the energy consuming part, but the human equivalent of that would be the (much more expensive!) years and years of learning and education of all sorts needed before becoming a competent adult. Do we have any stronger arguments to believe that there are theoretically much lower (or no bounds at all!) to training even a full AGI?
I think there are lower bounds in practice. Unsure if in theory. My personal estimate (which I outline in a report I’ve decided not to share because it gives too clear a roadmap), based on extensive research and months of careful thought and discussion with multiple experts, is that the practical lower bound on current technology is somewhere around 5-10k USD. I expect this practical lower bound to drop by a factor of about 10x over the next 5 years.
That’s a lower bound given a very high level of algorithmic improvements. A more moderate level of algorithmic improvements would get you to something more like 100-200k USD.
Here’s one insight for you. The human brain is forced to start from mostly scratch, just a bunch of genetically-hardwired long range connections with random local connections. Neural networks can be initialized. The better we are able to interpret and distill existing large powerful neural nets, the more easily we can cheaply initialize smaller cheaper neural nets to jump-start them. Look at the recent news around the Alpaca model. That’s just the beginning. This paradigm can be taken further.
Are AI scientists that you know in a pursuit for AGI or more powerful narrow AI systems?
As someone who is knew to this space I’m trying to simply wrap my head around the desire to create AGI, which could be intensely frightening and dangerous to the developer of such system.
I mean not that many people are hell bent on finding the next big virus or developing the next weapon so I don’t see why AGI is as inevitable as you say it is. Thus I suppose developers of these systems must have a firm belief there are very little dangers attached to developing a system some 2-5x general human intelligence.
If you happen to be one of these developers could you perhaps share with me the thesis behind why you feel this way or at least the studies, papers, etc that gives you assurance what you’re doing is largely beneficial to society as a whole and safe.
There are a lot of groups pursuing AGI. Some claiming that they are doing so with the goal of benefiting humanity, some simply in pursuit of profit and power. Indeed, the actors I personally am most concerned about are those who are relatively selfish and immoral as well as self-confident and incautious, and sufficiently competent to at least utilize and modify code published by researchers. Those who think they can dodge or externalize-to-society the negative consequences and reap the benefits, who don’t take the existential risk stuff seriously. You know what I mean. The L33T |-|ACKZ0R demographic.
I don’t personally work in AI. But Open AI for example states clearly in its own goals that they aim at building AGI, and Sam Altman wrote a whole post called “Moore’s Law for Everything” in which he outlines his vision for an AGI future. I consider it naïve nonsense, personally, but the drive seems to be simply the idea of a utopian world of abundance and technological development going faster and faster as AGI makes itself smarter.
EDIT: sorry, didn’t realise you weren’t answering to me, so my answer doesn’t make a lot of sense. Still, gonna leave it here.
actually I think I’ll name one.
@Nathan Helm-Burger thoughts? I ping you rather than the others because I expect you to be by far the most careful of the ones I know about.
Thanks Gears. Yeah, I’m quite confident that we cannot rely on scaling being a mandatory part of advances in order to monitor the AGI landscape. I believe dangerously potent advances can come from small research groups working with garage-hobbyist compute. I also think that scaling without algorithmic innovation can also lead to dangerously potent advances. And some combination of the two can also work. There are many paths to AGI, and we can’t effectively ward against it by guarding just the most expensive and obvious paths while ignoring the darker quieter paths out of the limelight.
Can I get some insights on this? My feeling is that while the human brain runs on very little energy, so does a LLM in operation, right now. The training of an LLM is the energy consuming part, but the human equivalent of that would be the (much more expensive!) years and years of learning and education of all sorts needed before becoming a competent adult. Do we have any stronger arguments to believe that there are theoretically much lower (or no bounds at all!) to training even a full AGI?
I think there are lower bounds in practice. Unsure if in theory. My personal estimate (which I outline in a report I’ve decided not to share because it gives too clear a roadmap), based on extensive research and months of careful thought and discussion with multiple experts, is that the practical lower bound on current technology is somewhere around 5-10k USD. I expect this practical lower bound to drop by a factor of about 10x over the next 5 years.
That’s a lower bound given a very high level of algorithmic improvements. A more moderate level of algorithmic improvements would get you to something more like 100-200k USD.
Here’s one insight for you. The human brain is forced to start from mostly scratch, just a bunch of genetically-hardwired long range connections with random local connections. Neural networks can be initialized. The better we are able to interpret and distill existing large powerful neural nets, the more easily we can cheaply initialize smaller cheaper neural nets to jump-start them. Look at the recent news around the Alpaca model. That’s just the beginning. This paradigm can be taken further.
Are AI scientists that you know in a pursuit for AGI or more powerful narrow AI systems?
As someone who is knew to this space I’m trying to simply wrap my head around the desire to create AGI, which could be intensely frightening and dangerous to the developer of such system.
I mean not that many people are hell bent on finding the next big virus or developing the next weapon so I don’t see why AGI is as inevitable as you say it is. Thus I suppose developers of these systems must have a firm belief there are very little dangers attached to developing a system some 2-5x general human intelligence.
If you happen to be one of these developers could you perhaps share with me the thesis behind why you feel this way or at least the studies, papers, etc that gives you assurance what you’re doing is largely beneficial to society as a whole and safe.
There are a lot of groups pursuing AGI. Some claiming that they are doing so with the goal of benefiting humanity, some simply in pursuit of profit and power. Indeed, the actors I personally am most concerned about are those who are relatively selfish and immoral as well as self-confident and incautious, and sufficiently competent to at least utilize and modify code published by researchers. Those who think they can dodge or externalize-to-society the negative consequences and reap the benefits, who don’t take the existential risk stuff seriously. You know what I mean. The L33T |-|ACKZ0R demographic.
I don’t personally work in AI. But Open AI for example states clearly in its own goals that they aim at building AGI, and Sam Altman wrote a whole post called “Moore’s Law for Everything” in which he outlines his vision for an AGI future. I consider it naïve nonsense, personally, but the drive seems to be simply the idea of a utopian world of abundance and technological development going faster and faster as AGI makes itself smarter.
EDIT: sorry, didn’t realise you weren’t answering to me, so my answer doesn’t make a lot of sense. Still, gonna leave it here.