In the short term: moderately big deal. The chip industry is currently in rather a lot of flux; Intel was supplanted as leader in transistor size by TSMC; Apple is running with their own chip designs; China’s monopoly on rare earth mineral processing has come under scrutiny again. This has provoked a boom in new development as a consequence. Even a small improvement in the design and manufacture of these facilities weighs a lot; because the chip industry is so important and so centralized, moderately big deal is essentially the floor for any actual development within it.
In the long term: big deal. This is not an opinion shared by anyone else as far as I can tell, but it feels very clear to me that “people use ML for this application” is the threshold at which the hardware overhang is almost immediately accessible to AGI. At that point the AGI is literally an upgrade operation, as opposed to having to go through the entire process of converting a workflow to something an AI of any type at all can work on. To be more concrete: I expect any kind of AI-driven takeover to control all of the currently-uses-ML industries before taking over any that do not; and I expect that within currently-uses-ML industries the order will be determined largely by how saturated they are with tools of that kind.
Thanks! So would you agree with my suggestion in the comment above that this will lower barriers to entry and allow many new chipmakers into the market?
I’m not yet convinced re AGI takeover. I feel like this sort of chip optimization is the sort of thing more suited to narrow AI than to AGI. Maybe in the long run additional optimizations could be gleaned by drawing on general world-knowledge that includes e.g. understanding of fractals and biological systems and city planning and so forth, but I feel like that would be only marginally better than what a narrow AI trained on simulated chips would produce.
I would agree; anything that cuts months and potentially hundreds of people makes it easier for new entrants. Further, the trend appears strongly in the direction of outsourcing, as even Intel will now build other’s designs. I see no reason why this could not be done on a contracting basis as well. The primary obstacle is the low appetite for the private sector to make large investments in physical things. Intel and TSMC’s new investments are largely defense motivated.
I agree that this particular sort of chip optimization is suited more for narrow AI than AGI; my claim is rather that anything which employs narrow AI is more vulnerable to AGI takeover. It seems likely to me that AGI would have an interest in production of processing power, so it seems like automating the steps is lowering the threshold.
I also consider that this kind of development is exactly what the CAIS model predicts. If CAIS is a system of narrow AIs, including coordinator/management AIs, why won’t a misaligned or malevolent coordinator AI from interacting with already existing narrow AIs? The malevolent case could be as straightforward as an ML redux of Stuxnet.
All of this rests pretty heavily on the crux that once one AI runs a task, it is easy to replace it with another AI; if this effect is weak, or I am completely wrong and it is in fact harder, then the chain of logic falls apart.
I see this as analogous to the points you made in the embodied intellectual property post comments: what we think we are doing is making more efficient use of resources, but what we are actually doing is engaging in a tradeoff of gaining time and money in exchange for living with a more opaque method of controlling the work. Within this more opaque method, additional risks lie. A more specific analogy to the Portuguese sailing technology commentary in the Conquistadors post feels achievable, but it isn’t coming together for me yet.
In the short term: moderately big deal. The chip industry is currently in rather a lot of flux; Intel was supplanted as leader in transistor size by TSMC; Apple is running with their own chip designs; China’s monopoly on rare earth mineral processing has come under scrutiny again. This has provoked a boom in new development as a consequence. Even a small improvement in the design and manufacture of these facilities weighs a lot; because the chip industry is so important and so centralized, moderately big deal is essentially the floor for any actual development within it.
In the long term: big deal. This is not an opinion shared by anyone else as far as I can tell, but it feels very clear to me that “people use ML for this application” is the threshold at which the hardware overhang is almost immediately accessible to AGI. At that point the AGI is literally an upgrade operation, as opposed to having to go through the entire process of converting a workflow to something an AI of any type at all can work on. To be more concrete: I expect any kind of AI-driven takeover to control all of the currently-uses-ML industries before taking over any that do not; and I expect that within currently-uses-ML industries the order will be determined largely by how saturated they are with tools of that kind.
Thanks! So would you agree with my suggestion in the comment above that this will lower barriers to entry and allow many new chipmakers into the market?
I’m not yet convinced re AGI takeover. I feel like this sort of chip optimization is the sort of thing more suited to narrow AI than to AGI. Maybe in the long run additional optimizations could be gleaned by drawing on general world-knowledge that includes e.g. understanding of fractals and biological systems and city planning and so forth, but I feel like that would be only marginally better than what a narrow AI trained on simulated chips would produce.
I would agree; anything that cuts months and potentially hundreds of people makes it easier for new entrants. Further, the trend appears strongly in the direction of outsourcing, as even Intel will now build other’s designs. I see no reason why this could not be done on a contracting basis as well. The primary obstacle is the low appetite for the private sector to make large investments in physical things. Intel and TSMC’s new investments are largely defense motivated.
I agree that this particular sort of chip optimization is suited more for narrow AI than AGI; my claim is rather that anything which employs narrow AI is more vulnerable to AGI takeover. It seems likely to me that AGI would have an interest in production of processing power, so it seems like automating the steps is lowering the threshold.
I also consider that this kind of development is exactly what the CAIS model predicts. If CAIS is a system of narrow AIs, including coordinator/management AIs, why won’t a misaligned or malevolent coordinator AI from interacting with already existing narrow AIs? The malevolent case could be as straightforward as an ML redux of Stuxnet.
All of this rests pretty heavily on the crux that once one AI runs a task, it is easy to replace it with another AI; if this effect is weak, or I am completely wrong and it is in fact harder, then the chain of logic falls apart.
I see this as analogous to the points you made in the embodied intellectual property post comments: what we think we are doing is making more efficient use of resources, but what we are actually doing is engaging in a tradeoff of gaining time and money in exchange for living with a more opaque method of controlling the work. Within this more opaque method, additional risks lie. A more specific analogy to the Portuguese sailing technology commentary in the Conquistadors post feels achievable, but it isn’t coming together for me yet.