I think that convincing Chinese researchers and policy-makers of the importance of the alignment problem would be very valuable, but it also risks changing the focus of race dynamics to AGI, and is therefore very risky. The last thing you want to do is leave the CCP convinced that AGI is very important but safety isn’t, as happened to John Carmack! Also beware thinking you’re in a race.
I think (3) is only true over timescales of 2--5 years: the thing that really matters is performance per unit cost, and if you own the manufacturer you’re not paying Nvidia’s ~80% unit margins.
I think that convincing Chinese researchers and policy-makers of the importance of the alignment problem would be very valuable, but it also risks changing the focus of race dynamics to AGI, and is therefore very risky. The last thing you want to do is leave the CCP convinced that AGI is very important but safety isn’t, as happened to John Carmack! Also beware thinking you’re in a race.
Unfortunately for us LessWrongers, it’s probably rational for them to believe that a race is happening, because to put it bluntly, China having AGI would become the world power that exceeds even the US. Problem is, the US government doesn’t care about the alignment problem.
I think that convincing Chinese researchers and policy-makers of the importance of the alignment problem would be very valuable, but it also risks changing the focus of race dynamics to AGI, and is therefore very risky. The last thing you want to do is leave the CCP convinced that AGI is very important but safety isn’t, as happened to John Carmack! Also beware thinking you’re in a race.
I think (3) is only true over timescales of 2--5 years: the thing that really matters is performance per unit cost, and if you own the manufacturer you’re not paying Nvidia’s ~80% unit margins.
Unfortunately for us LessWrongers, it’s probably rational for them to believe that a race is happening, because to put it bluntly, China having AGI would become the world power that exceeds even the US. Problem is, the US government doesn’t care about the alignment problem.