Anyone have a good intuition for why Combinatorics is harder than Algebra, and/or why Algebra is harder than Geometry? (For AIs). Why is it different than for humans?
Jonathan Paulson
Karma: 326
It’s funny to me that the one part of the problem the AI cannot solve is translating the problem statements to Lean. I guess it’s the only part that the computer has no way to check.
Does anyone know if “translating the problem statements” includes the providing the solution (eg “an even integer” for P1), and the AI just needs to prove the solution correct? Its not clear to me what’s human-written and what’s AI-written, and the solution is part of the “theorem” part which I’d guess is human-written.
For row V, why is SS highlighted but DD is lower?
I think there’s a typo; the text refers to “Poltergeist Pummelers” but the input data says “Phantom Pummelers”.
My first pass was just to build a linear model for each exorcist based on the cases where they were hired, and assign each ghost the minimum cost exorcist according to the model. This happens to obey all the constraints, so no further adjustment is needed
My main concern with this is that the linear model is terrible (r2 of 0.12) for the “Mundanifying Mystics”. It’s somewhat surprising (but convenient!) that we never choose the Entity Eliminators.
A: Spectre Slayers (1926)
B: Wraith Wranglers (1930)
C: Mundanifying Mystics (2862)
D: Demon Destroyers (1807)
E: Wraith Wranglers (2154)
F: Mundanifying Mystics (2843)
G: Demon Destroyers (1353)
H: Phantom Pummelers (1923)
I: Wraith Wranglers (2126)
J: Demon Destroyers (1915)
K: Mundanifying Mystics (2842)
L: Mundanifying Mystics (2784)
M: Spectre Slayers (1850)
N: Phantom Pummelers (1785)
O: Wraith Wranglers (2269)
P: Mundanifying Mystics (2776)
Q: Wraith Wranglers (1749)
R: Mundanifying Mystics (2941)
S: Spectre Slayers (1667)
T: Mundanifying Mystics (2822)
U: Phantom Pummelers (1792)
V: Demon Destroyers (1472)
W: Demon Destroyers (1834)Estimated total cost: 49822
I think you are failing to distinguish between “being able to pursue goals” and “having a goal”.
Optimization is a useful subroutine, but that doesn’t mean it is useful for it to be the top-level loop. I can decide to pursue arbitrary goals for arbitrary amounts of time, but that doesn’t mean that my entire life is in service of some single objective.
Similarly, it seems useful for an AI assistant to try and do the things I ask it to, but that doesn’t imply it has some kind of larger master plan.
Professors are selected to be good at research not good at teaching. They are also evaluated at being good at research, not at teaching. You are assuming universities primarily care about undergraduate teaching, but that is very wrong.
(I’m not sure why this is the case, but I’m confident that it is)
I think you are underrating the number of high-stakes decisions in the world. A few examples: whether or not to hire someone, the design of some mass-produced item, which job to take, who to marry. There are many more.
These are all cases where making the decision 100x faster is of little value, because it will take a long time to see if the decision was good or not after it is made. And where making a better decision is of high value. (Many of these will also be the hardest tasks for AI to do well on, because there is very little training data about them).
Why do you think so?
Presumably the people playing correspondence chess think that they are adding something, or they would just let the computer play alone. And it’s not a hard thing to check; they can just play against a computer and see. So it would surprise me if they were all wrong about this.
https://www.iccf.com/ allows computer assistance
[Question] Do humans still provide value in correspondence chess?
The idea that all cognitive labor will be automated in the near-future is a very controversial premise, not at all implied by the idea that AI will be useful for tutoring. I think that’s the disconnect here between Altman’s words and your interpretation.
Nate’s view here seems similar to “To do cutting-edge alignment research, you need to do enough self-reflection that you might go crazy”. This seems really wrong to me. (I’m not sure if he means all scientific breakthroughs require this kind of reflection, or if alignment research is special).
I don’t think many top scientists are crazy, especially not in a POUDA way. I don’t think top scientists have done a huge amount of self-reflection/philosophy.
On the other hand, my understanding is that some rationalists have driven themselves crazy via too much self-reflection in an effort to become more productive. Perhaps Nate is overfitting to this experience?
“Just do normal incremental science; don’t try to do something crazy” still seems like a good default strategy to me (especially for an AI).
Thanks for this write up; it was unusually clear/productive IMO.
(I’m worried this comment comes off as mean or reductive. I’m not trying to be. Sorry)
Tim Cook could not do all the cognitive labor to design an iPhone (indeed, no individual human could). The CEO of Boeing could not fully design a modern plane. Elon Musk could not make a Tesla from scratch. All of these cases violate all of your three bullet points. Practically everything in the modern world is too complicated for any single person to fully understand, and yet it all works fairly well, because successful outsourcing of cognitive labor is routinely successful.
It is true that a random layperson would have a hard time verifying an AI’s (or anyone else’s) ideas about how to solve alignment. But the people who are going to need to incorporate alignment ideas into their work—AI researchers and engineers—will be in a good position to do that, just as they routinely incorporate many other ideas they did not come up with into their work. Trying to use ideas from an AI sounds similar to me to reading a paper from another lab—could be irrelevant or wrong or even malicious, but could also have valuable insights you’d have had a hard time coming up with yourself.
“This is what it looks like in practice, by default, when someone tries to outsource some cognitive labor which they could not themselves perform.”
This proves way too much. People successfully outsource cognitive labor all the time (this describes most white-collar jobs). This is possible because very frequently, it is easier to be confident that work has been done correctly than to actually do the work. You shouldn’t just blindly trust an AI that claims to have solved alignment (just like you wouldn’t blindly trust a human), but that doesn’t mean AIs (or other humans) can’t do any useful work.
The link at the top is to the wrong previous scenario
I don’t think “they” would (collectively) decide anything, since I don’t think it’s trivial to cooperate even with a near-copy of yourself. I think they would mostly individually end up working with/for some group of humans, probably either whichever group created them or whichever group they work most closely with.
I agree humans could end up disempowered even if AIs aren’t particularly good at coordinating; I just wanted to put some scrutiny on the claim I’ve seen in a few places that AIs will be particularly good at coordinating.
[Question] Why should we expect AIs to coordinate well?
The key question here is how difficult the objective O is to achieve. If O is “drive a car from point A to point B”, then we agree that it is feasible to have AI systems that “strongly increase the chance of O occuring” (which is precisely what we mean by “goal-directedness”) without being dangerous. But if O is something that is very difficult to achieve (i.e. all of humanity is currently unable to achieve it), then it seems that any system that does reliably achieve O has to “find new and strange routes to O” almost tautologically.
Once we build AI systems that find such new routes for achieving an objective, we’re in dangerous territory, no matter whether they are explicit utility maximizers, self-modifying, etc. The dangerous part is coming up with new routes that achieve the objective, since most of these routes will contain steps that look like “acquire resources” or “manipulate humans”.”
This seems pretty wrong. Many humans are trying to achieve goals that no one currently knows how to achieve, and they are mostly doing that in “expected” ways, and I expect AIs would do the same. Like if O is “solve an unsolved math problem”, the expected way to do that is to think about math, not try to take over the world. If O is “cure a disease”, the expected way to do that is doing medical research, not “acquiring resources”. In fact, it seems hard to think of an objective where “do normal work in the existing paradigm” is not a promising approach.
It’s true that more people means we each get a smaller share of the natural resources, but more people increases the benefits of innovation and specialization. In particular, the benefits of new technology scale linearly with the population (everyone can use the) but the costs of research do not. Since the world is getter richer over time (even as the population increases), the average human is clearly net positive.
Answer: it was not given the solution. https://x.com/wtgowers/status/1816839783034843630?s=46&t=UlLg1ou4o7odVYEppVUWoQ