Targeting ML engineers (aiming to convince them that this is a problem):
It’s hard to ask optimizers the right questions; you may not get what you want. Capitalism is a powerful optimizer. We asked it for cheap transportation and we got Global Warming. In the coming decades we will build AI with far more optimizing power than capitalism, and we don’t know how to ask it the right questions.
In the past, capitalism did these things while optimizing:
Put cocaine in coca cola
Got almost everyone addicted to tobbacco
Made billions of people waste trillions of dollars on ludicrously overpriced and unnecessary shoes and clothes, in order to maintain a basic level of competitive appearance-upkeep with their friends
Put dangerous levels of sugar in most bread, coffee, and restaurant food
Built apartment buildings and planes with ventilation systems that spread disease, including COVID
Communist countries did this too by copying the technology, and it persisted with alcohol during prohibition, so it’s optimization itself that does these horrible things, not any particular culture or ideology.
Oh sure. I was pointing to capitalism as one of the stronger optimizers humans regularly use. It’s certainly not the only one.
FWIW I also like it as an example because there’s a lot of cultural awareness of the failure modes and an intuition that they come from optimizing the wrong thing. So you don’t need to do as much work to convince people because the examples are already salient.
Targeting ML engineers (aiming to convince them that this is a problem):
In the past, capitalism did these things while optimizing:
Put cocaine in coca cola
Got almost everyone addicted to tobbacco
Made billions of people waste trillions of dollars on ludicrously overpriced and unnecessary shoes and clothes, in order to maintain a basic level of competitive appearance-upkeep with their friends
Put dangerous levels of sugar in most bread, coffee, and restaurant food
Built apartment buildings and planes with ventilation systems that spread disease, including COVID
Communist countries did this too by copying the technology, and it persisted with alcohol during prohibition, so it’s optimization itself that does these horrible things, not any particular culture or ideology.
Oh sure. I was pointing to capitalism as one of the stronger optimizers humans regularly use. It’s certainly not the only one.
FWIW I also like it as an example because there’s a lot of cultural awareness of the failure modes and an intuition that they come from optimizing the wrong thing. So you don’t need to do as much work to convince people because the examples are already salient.