Modern self driving vehicles can’t run inference on even a chinchilla scale network locally in real time, latency and reliability requirements preclude most server-side work, and even if you could use big servers to help, it costs a lot of money to run large models for millions of customers simultaneously.
This is a good point regarding latency.
Why wouldn’t it also apply to a big datacenter? If it’s a few hundred meters of distance from the two farthest apart processing units, that seems to imply an enormous latency in computing terms.
Latency only matters to the degree that something is waiting on it. If your car won’t respond to an event until a round trip across a wireless connection, and oops dropped packet, you’re not going to have a good time.
In a datacenter, not only are latencies going to be much lower, you can often set things up that you can afford to wait for whatever latency remains. This is indeed still a concern- maintaining high utilization while training across massive numbers of systems does require hard work- but that’s a lot different than your car being embedded in a wall.
This is a good point regarding latency.
Why wouldn’t it also apply to a big datacenter? If it’s a few hundred meters of distance from the two farthest apart processing units, that seems to imply an enormous latency in computing terms.
Latency only matters to the degree that something is waiting on it. If your car won’t respond to an event until a round trip across a wireless connection, and oops dropped packet, you’re not going to have a good time.
In a datacenter, not only are latencies going to be much lower, you can often set things up that you can afford to wait for whatever latency remains. This is indeed still a concern- maintaining high utilization while training across massive numbers of systems does require hard work- but that’s a lot different than your car being embedded in a wall.