Modern technologies are getting increasingly complicated… but when you get down to it, a car is just a box with wheels and a combustion engine. There aren’t that many ways for a outcome-perception-driven AI to go “oops, I accidentally concealed a human-killing machine gun inside the steering wheel!”, especially if the AI has to subcontract to independent suppliers for parts.
Moreover, tight constraints. Such a machine gun adds weight and cost without benefit to the AIs reward heuristic. A far more likely problem is it removes structure somewhere because every collision test doesn’t need that material to pass. But the missing structure causes fatalities in crashes a conservatively designed vehicle would survive or long term durability problems.
Human designed products have exactly this happen also however. The difference is you could make a patch to add another reward heuristic component and have another design in the prototyping phase that same week. It would let you move fast and break things and fix them far faster than human organizations can.
Yeah, that was my initial reaction as well.
Modern technologies are getting increasingly complicated… but when you get down to it, a car is just a box with wheels and a combustion engine. There aren’t that many ways for a outcome-perception-driven AI to go “oops, I accidentally concealed a human-killing machine gun inside the steering wheel!”, especially if the AI has to subcontract to independent suppliers for parts.
Moreover, tight constraints. Such a machine gun adds weight and cost without benefit to the AIs reward heuristic. A far more likely problem is it removes structure somewhere because every collision test doesn’t need that material to pass. But the missing structure causes fatalities in crashes a conservatively designed vehicle would survive or long term durability problems.
Human designed products have exactly this happen also however. The difference is you could make a patch to add another reward heuristic component and have another design in the prototyping phase that same week. It would let you move fast and break things and fix them far faster than human organizations can.