What does it look like, when the optimization power is turned up to 11 on something like the air conditioner problem?
I think it looks exactly like it does now; with a lot of people getting very upset that local optimization often looks un-optimized from the global perspective.
If I needed an air-conditioner for working in my attic space, which is well-insulated from my living space and much, much hotter than either my living space or the outside air in the summer, the single-vent model would be more efficient. Indeed, it is effectively combining the modern technology of an air conditioner, with the much older technology of what is sometimes called a “house fan”—a literal giant fan that pulled air from the outside, inside, which obviously only works when the interior is hotter than the exterior, conditions under which a single-vent AC should actually outperform the double-hose model.
I know somebody who uses one of the one-hose models. It is run in an otherwise entirely un-air-conditioned space, which is hotter than the outside air when the AC is not in action. Like my attic space, it is more efficient than the two-hose model would be, because the infiltration effect -cools the space-, owing to the fact that interior spaces here are generally hotter than exterior spaces.
Now, I have a room AC unit; it’s a two-hose model, because it is air conditioning a single room in an air-conditioned space. However, I have been tempted to install an AC in a different room once before—a room which, owing to a combination of poor insulation and airflow, was considerably warmer than the rest of the house. From the perspective of cooling -that room-, the infiltration effect would be a net positive, as the infiltrating air is cooler than the “natural tendency” of the room; from the perspective of cooling -the entire house-, it would be a net negative, relative to a double-hose model. Does this mean a single-hose model is always worse for this application? Well—that depends on the relative efficiency of the small AC unit compared to the central air conditioning working in the rest of the house. Assuming the central AC is more efficient at cooling air than the room AC, which seems like a reasonable bet, then infiltration could, as with the case for environments which are hotter than the exterior, actually result in a net increase in efficiency under some operating conditions.
Now, just as your post is kind of an analogy about optimization failures, this response is kind of an analogy about optimization failures—namely, the kind of optimization failure where you optimize for a single situation, and end up making things worse.
And namely, I trust local optimizations more than I trust global optimizations, which means in an important sense that I think alignment, as a form of global optimization, may be significantly worse in important ways than “non-aligned” intelligence. Fortunately for this perspective, I expect “aligned” intelligence is necessarily worse at intelligence than non-aligned intelligence, and indeed general-purpose intelligence to always be worse than specific-purpose intelligence, and by such a factor that I think we gravely miscalculate how hard general intelligence is, because we base our predictions on how successful we are at creating specific-purpose intelligence.
There’s a kind of general relationship here between “general purpose” and “global”, and “specific purpose” and “local”, which I believe should hold up under scrutiny.
ETA:
One thing to pay particular attention to, in the efficiency assumptions, is the following question: Is the device expected to be “always on”? There’s a huge difference between a device whose primary purpose is effectively to “Cool off a relatively hot space, and then maintain that temperature for some duration of time” and a device whose primary function is effectively to “Maintain an existing temperature differential indefinitely (with a negligible consideration for the ramp-up period)”. Of note, basically everybody I know who owns such a device is not keeping it on indefinitely, and the ramp-up period likely dominates the energy consumption of the device.
I think it looks exactly like it does now; with a lot of people getting very upset that local optimization often looks un-optimized from the global perspective.
If I needed an air-conditioner for working in my attic space, which is well-insulated from my living space and much, much hotter than either my living space or the outside air in the summer, the single-vent model would be more efficient. Indeed, it is effectively combining the modern technology of an air conditioner, with the much older technology of what is sometimes called a “house fan”—a literal giant fan that pulled air from the outside, inside, which obviously only works when the interior is hotter than the exterior, conditions under which a single-vent AC should actually outperform the double-hose model.
I know somebody who uses one of the one-hose models. It is run in an otherwise entirely un-air-conditioned space, which is hotter than the outside air when the AC is not in action. Like my attic space, it is more efficient than the two-hose model would be, because the infiltration effect -cools the space-, owing to the fact that interior spaces here are generally hotter than exterior spaces.
Now, I have a room AC unit; it’s a two-hose model, because it is air conditioning a single room in an air-conditioned space. However, I have been tempted to install an AC in a different room once before—a room which, owing to a combination of poor insulation and airflow, was considerably warmer than the rest of the house. From the perspective of cooling -that room-, the infiltration effect would be a net positive, as the infiltrating air is cooler than the “natural tendency” of the room; from the perspective of cooling -the entire house-, it would be a net negative, relative to a double-hose model. Does this mean a single-hose model is always worse for this application? Well—that depends on the relative efficiency of the small AC unit compared to the central air conditioning working in the rest of the house. Assuming the central AC is more efficient at cooling air than the room AC, which seems like a reasonable bet, then infiltration could, as with the case for environments which are hotter than the exterior, actually result in a net increase in efficiency under some operating conditions.
Now, just as your post is kind of an analogy about optimization failures, this response is kind of an analogy about optimization failures—namely, the kind of optimization failure where you optimize for a single situation, and end up making things worse.
And namely, I trust local optimizations more than I trust global optimizations, which means in an important sense that I think alignment, as a form of global optimization, may be significantly worse in important ways than “non-aligned” intelligence. Fortunately for this perspective, I expect “aligned” intelligence is necessarily worse at intelligence than non-aligned intelligence, and indeed general-purpose intelligence to always be worse than specific-purpose intelligence, and by such a factor that I think we gravely miscalculate how hard general intelligence is, because we base our predictions on how successful we are at creating specific-purpose intelligence.
There’s a kind of general relationship here between “general purpose” and “global”, and “specific purpose” and “local”, which I believe should hold up under scrutiny.
ETA:
One thing to pay particular attention to, in the efficiency assumptions, is the following question: Is the device expected to be “always on”? There’s a huge difference between a device whose primary purpose is effectively to “Cool off a relatively hot space, and then maintain that temperature for some duration of time” and a device whose primary function is effectively to “Maintain an existing temperature differential indefinitely (with a negligible consideration for the ramp-up period)”. Of note, basically everybody I know who owns such a device is not keeping it on indefinitely, and the ramp-up period likely dominates the energy consumption of the device.