Vitor

Karma: 822

Vitor Jan 4, 2023, 4:04 PM
17 points
11
on: What’s up with ChatGPT and the Turing Test?
I’m not buying the premise. Passing the Turing test requires to fool an alert, smart person who is deliberately probing the limits of the system. ChatGPT isn’t at that level.

A specially tuned persona that is optimized for this task might do better than the “assistant” persona we have available now, but the model is currently incapable of holding a conversation without going on long, unwanted tangents, getting trapped in loops, etc.

Vitor Dec 29, 2022, 3:03 AM
1 point
0
on: Rock-Paper-Scissors Can Be Weird
The name I have in my head for this is “zones of control”. In board- and videogames, sometimes a unit explicitly has an effect on tiles adjacent to its own. I expanded the term from there to include related phenomena, for example where the mere existence of strategy X blocks strategy Y from ever being played, even if X itself is almost never played either. X is in some sense providing “cover fire”, not achieving anything directly, but pinning down another strategy in the process.

This case doesn’t match that intuition exactly, but it’s in the same neighborhood.

Vitor Dec 23, 2022, 5:53 PM
2 points
0
in reply to: sanxiyn’s comment on: Let’s think about slowing down AI
The difference between regulation and research is that the former has a large amount of friction, making it about as hard to push a 1% regulation through as a 10% one.

In contrast, the incremental 1% improvements in the development of capabilities is just what happens by default, as research organizations follow their charter.

Vitor Dec 23, 2022, 5:48 PM
4 points
−8
in reply to: Scott Alexander’s comment on: Let’s think about slowing down AI
Agreed. My main objection to the post is that it considers the involved agents to be optimizing for far future world-states. But I’d say that most people (including academics and AI lab researchers) mostly only think of the next 1% step in front of their nose. The entire game theoretic framing in the arms race etc section seems wrong to me.

Vitor Dec 13, 2022, 3:20 PM
3 points
1
in reply to: Daniel Kokotajlo’s comment on: Updating my AI timelines
Mmm, I would say the general shape of your view won’t clash with reality, but the magnitude of the impact will.

It’s plausible to me that a smart buyer will go and find the best deal for you when you tell it to buy laptop model X. It’s not plausible to me that you’ll be able to instruct it “buy an updated laptop for me whenever a new model comes out that is good value and sufficiently better than what I already have,” and then let it do its thing completely unsupervised (with direct access to your bank account). That’s what I mean by multiple complicated objectives.

What counts as “domain where correctness matters?” What counts as “very constrained set of actions?” Would e.g. a language-model-based assistant that can browse the internet and buy things for you on Amazon (with your permission of course) be in line with what you expect, or violate your expectations?

Something that goes beyond current widespread use of AI such as spam-filtering. Spam-filtering (or selecting ads on facebook, or flagging hate speech etc) is a domain where the AI is doing a huge number of identical tasks, and a certain % of wrong decisions is acceptable. One wrong decision won’t tank the business. Each copy of the task is done in an independent session (no memory).

An example application where that doesn’t hold is putting the AI in charge of ordering all the material inputs for your factory. Here, a single stupid mistake (didn’t buy something because the price will go down in the future, replaced one product with another, misinterpret seasonal cycles) will lead to a catastrophic stop of the entire operation.

(Also, what about Copilot? Isn’t it already an example of an application that genuinely works, and isn’t just in the twilight zone?)

Copilot is not autonomous. There’s a human tightly integrated into everything it’s doing. The jury is still out on if it works, i.e., do we have anything more than some programmers’ self reports to substantiate that it increases productivity? Even if it does work, it’s just a productivity tool for humans, not something that replaces humans at their tasks directly.

Vitor Dec 13, 2022, 1:10 AM
5 points
0
in reply to: Daniel Kokotajlo’s comment on: Updating my AI timelines

OK, well, you should retract your claim that the median LW timeline will soon start to clash with reality then! It sounds like you think reality will look basically as I predicted! (I can’t speak for all of LW of course but I actually have shorter timelines than the median LWer, I think.)

I retract the claim in the sense that it was a vague statement that I didn’t expect to be taken literally, which I should have made clearer! But it’s you who operationalized “a few years” as 2026 and “the median less wrong view” as your view.

Anyway, I think I see the outline of our disagreement now, but it’s still kind of hard to pin down.

First, I don’t think that AIs will be put to unsupervised use in any domain where correctness matters, i.e., given fully automated access to valuable resources, like money or compute infrastructure. The algorithms that currently do this have a very constrained set of actions they can take (e.g. an AI chooses an ad to show out of a database of possible ads), and this will remain so.

Second, perhaps I didn’t make clear enough that I think all of the applications will remain in this twilight of almost working, showing some promise, etc, but not actually deployed (that’s what I meant by the economic impact remaining small). So, more thinkpieces about what could happen (with isolated, splashy examples), rather than things actually happening.

Third, I don’t think AIs will be capable of performing tasks that require long attention spans, or that trade off multiple complicated objectives against each other. With current technology, I see AIs constrained to be used for short, self-contained tasks only, with a separate session for each task.

Does that make the disagreement clearer?

Vitor Dec 9, 2022, 11:12 PM
5 points
−1
in reply to: Daniel Kokotajlo’s comment on: Updating my AI timelines
I do roughly agree with your predictions, except that I rate the economic impact in general to be lower. Many headlines, much handwringing, but large changes won’t materialize in a way that matters.

To put my main objection succinctly, I simply don’t see why AGI would follow soon from your 2026 world. Can you walk me through it?

Vitor Dec 9, 2022, 12:46 AM
15 points
4
in reply to: Daniel Kokotajlo’s comment on: Updating my AI timelines
Sure, let me do this as an exercise (ep stat: babble mode). Your predicions are pretty sane overall, but I’d say you handwave away problems (like integration over a variety of domains, long-term coherent behavior, and so on) that I see as (potentially) hard barriers to progress.

2022
- 2022 is basically over and I can’t get a GPT instance to order me a USB stick online.
2023
- basically agree, this is where we’re at right now (perhaps with the intensity turned down a notch)
2024
- you’re postulating that “It’s easy to make a bureaucracy and fine-tune it and get it to do some pretty impressive stuff, but for most tasks it’s not yet possible to get it to do OK all the time.” I have a fundamental disagreement here. I don’t think these tools will be effective at doing any task autonomously (fooling other humans doesn’t count, neither does forcing humans to only interact with a company through one of these). Currently (2022) chatGPT is arguably useful as a babbling tool, stimulating human creativity and allowing it to make templating easier (this includes things like easy coding tasks). I don’t see anything in your post that justifies the implicit jump in capabilities you’ve snuck in here.
- broadly agree with your ideas on propaganda, from the production side (i.e. that lots of companies/governments will be doing lots of this stuff). But I think that general attitudes in the population will shift (cynicism etc) and provide some amount of herd immunity. Note that the influence of the woke movement is already fading, shortly after it went truly mainstream and started having visible influence in average people’s lives. This is not a coincidence.
2025
- Doing well at diplomacy is not very related to general reasoning skills. I broadly agree with Zvi’s take and also left some of my thoughts there.
- I’m very skeptical that bureaucracies will be the way forward. They work for trivial tasks but reliably get lost in the weeds and start talking to themselves in circles for anything requiring a non-trivial amount of context.
- disagree on orders of magnitude improvements in hardware. You’re proposing a 100x decrease in costs compared to 2020, when it’s not even clear our civilization is capable of keeping hardware at current levels generally available, let alone cope with a significant increase in demand. Semiconductor production is much more centralized/fragile than people think, so even though billions of these things are produced per year, the efficient market hypothesis does not apply to this domain.
2026
- Here you’re again postulating jumps in capabilities that I don’t see justified. You talk about the “general understanding and knowledge of pretrained transformers”, when understanding is definitely not there, and knowledge keeps getting corrupted by the AI’s tendency to synthesize falsities as confidently as truths. Insofar as the AI can be said to be intelligent at all, it’s all symbol manipulation at a high simulacron level. Integration with real-world tasks keeps mysteriously failing as the AI flounders around in a way that is simultaneously very sophisticated, but oh so very reminiscent of 2022.
- disagree about your thoughts on propaganda, which is just an obvious extension of my 2024 thoughts above. I also notice that social changes this large take orders of magnitude longer to percolate through society than what you predict, so I disagree with your predictions even conditioned on your views of the raw effectiveness of these systems.
- “chatbots quickly learn about themselves” etc. Here you’re conflating the regurgitation of desirable phrases with actual understanding. I notice that as you write your timeline, your language morphs to make your AIs more and more conscious, but you’re not justifying this in any way other than… something something self-referential, something something trained on their own arxiv papers. I don’t mean to be overly harsh, but here you seem to be sneaking in the very thing that’s under debate!

Vitor Dec 8, 2022, 2:54 PM
3 points
0
on: Why I’m Sceptical of Foom
This is reasonably close to my beliefs. An additional argument I’d like to add is:
- Even if superintelligence is possible, the economic path towards it might be impossible.
There needs to be an economically viable entity pushing AI development forward every step of the way. It doesn’t matter if AI can “eventually” produce 30% worldwide GPD growth. Maybe diminishing returns kick in around GPT-4, or we run out of useful training data to feed to the models (We have very few examples of +6 SD human reasoning, as MikkW points out in a sibling comment).

Analogy: It’s not the same to say that a given species with X,Y,Z traits can survive in an ecosystem, than to say it can evolve from its ancestor in that same ecosystem.

Vitor Dec 8, 2022, 2:44 PM
1 point
0
in reply to: Beth Barnes’s comment on: Using GPT-Eliezer against ChatGPT Jailbreaking
Nice! But we’re still missing a capability, namely causing the model to respond to a specific prompt, not just output an arbitrary unsafe thing.

Vitor Dec 8, 2022, 1:36 PM
3 points
0
on: Machine Learning Consent
How likely is it that this becomes a legal problem rendering models unable to be published? Note that using models privately (even within a firm) will always be an option, as copyright only applies to distribution of the work.

Vitor Dec 8, 2022, 1:34 PM
4 points
0
in reply to: the gears to ascension’s comment on: Machine Learning Consent
This is an interesting thought, but it seems very hard to realize as you have to distill the unique contribution of the sample, as opposed to much more widespread information that happens to be present in the sample.

Weight updates depend heavily on training order of course, so you’re really looking for something like the Shapley value of the sample, except that “impact” is liable to be an elusive, high-dimensional quantity in itself.

Vitor Dec 8, 2022, 1:29 PM
4 points
0
on: Updating my AI timelines
I also tend to find myself arguing against short timelines by default, even though I feel like I take AI safety way more seriously than most people.

At this point, how many people with long timelines are there still around here? I haven’t explicitly modeled mine, but it seems clear that they’re much, much longer (with significant weight on “never”) than the average less wronger. The next few years will for sure be interesting as we see the “median less wrong timeline” clash with reality.

Vitor Dec 2, 2022, 6:07 PM
2 points
0
in reply to: philip_b’s comment on: Jailbreaking ChatGPT on Release Day
For meth it lists an ingredient (ether) that it doesn’t actually use. And actual lab protocols are much more detailed about precise temperature, times, quantities, etc.

Vitor Nov 30, 2022, 9:32 PM
2 points
1
in reply to: green_leaf’s comment on: On the Diplomacy AI
(ep stat: it’s hard to model my past beliefs accurately, but this is how I remember it)

I mean, it’s unsurprising now, but before that series of matches where AlphaStar won, it was impossible.

Maybe for you. But anyone who has actually played starcraft knows that it is a game that is (1) heavily dexterity capped, and (2) intense enough that you barely have time to think strategically. It’s all snap decisions and executing pre-planned builds and responses.

I’m not saying it’s easy to build a system that plays this game well. But neither is it paradigm-changing to learn that such a thing was achieved, when we had just had the news of alphago beating top human players. I do remember being somewhat skeptical of these systems working for RTS games, because the action space is huge, so it’s very hard to even write down a coherent menu of possible actions. I still don’t really understand how this is achieved.

When AlphaStar is capped by human ability and data availability, it’s still better than 99.8% of players, unless I’m missing something, so even if all a posteriori revealed non-intelligence-related advantages are taken away, it looks like there is still some extremely significant Starcraft-specialized kind of intelligence at play.

I haven’t looked into this in detail, so assuming the characterization in the article is accurate, this is indeed significant progress. But the 99.8% number is heavily misleading. The system was tuned to have an effective APM of 268, that’s probably top 5% of human players. Even higher if we assume that the AI never misclicks, and never misses any information that it sees. The latter implies 1-frame reaction times to scouting anything of strategic significance, which is a huge deal.

Vitor Nov 29, 2022, 9:41 PM
3 points
1
in reply to: green_leaf’s comment on: On the Diplomacy AI
I agree, and I don’t use this argument regarding arbitrary AI achievements.

But it’s very relevant when capabilities completely orthogonal to the AI are being sold as AI. The Starcraft example is more egregious, because AlphaStar had a different kind of access to the game state than a human has, which was claimed to be “equivalent” by the deepmind team. This resulted in extremely fine-grained control of units that the game was not designed around. Starcraft is partially a sport, i.e., a game of dexterity, concentration, and endurance. It’s unsurprising that a machine beats a human at that.

If you (generic you) are going to make an argument about how speed of execution, parallel communication and so on are game changers (specially in an increasingly online, API accessible world), then make that argument. But don’t dress it up with the supposed intelligence of the agent in question.

Vitor Nov 29, 2022, 1:38 PM
5 points
0
on: Geometric Rationality is Not VNM Rational
I find this example interesting but very weird. The couple is determining fairness by using “probability mass of happiness” as the unit of account. But it seems very natural to me to go one step further and adjust for the actual outcomes, investing more resources into the sub-agent that has worse luck.

I don’t know if this is technically doable (I foresee complications with asymmetric utility functions of the two sub-agents, where one is harder to satisfy than the other, or even just has more variance in outcomes), but I think such an adjustment should recover the VNM independence condition.

Vitor Nov 29, 2022, 12:49 PM
3 points
0
on: On the Diplomacy AI
This confirms the suspicions I had upon hearing the news: the diplomacy AI falls in a similar category than the Starcraft II AI (AlphaStar) we had a while back.

“Robot beats humans in a race.” Turns out the robot has 4 wheels and an internal combustion engine.
What links here?
- Vitor's comment on Updating my AI timelines by Matthew Barnett (Dec 9, 2022, 12:46 AM; 15 points)

Vitor Nov 6, 2022, 3:03 AM
2 points
1
on: Introduction to abstract entropy
While I think this post overall gives good intuition for the subject, it also creates some needless confusion.

Your concept of “abstract entropy” is just Shannon entropy applied to uniform distributions. Introducing Shannon entropy directly, while slightly harder, gives you a bunch of the ideas in this post more or less “for free”:
- Macrostates are just events and microstates are atomic outcomes (as defined in probability theory). Any rules how the two relate to each other follow directly from the foundations of probability.
- The fact that E[-log x] is the only reasonable function that can serve as a measure of information. You haven’t actually mentioned this (yet?), but having an axiomatic characterization of entropy hammers home that all of this stuff must be the way it is because of how probability works. For example, your “pseudo-entropy” of Rubik’s cube states (distance from the solved state) might be intuitive, but it is wrong!
- derived concepts, such as conditional entropy and mutual information, fall out naturally from the probabilistic view.
- the fact that an optimal bit/guess must divide the probability mass in half, not the probability space.
I hope I don’t come across as overly harsh. I know that entropy is often introduced in confusing ways in various sciences, specially physics, where its hopelessly intertwined with the concrete subject matter. But that’s definitely not the case in computer science (more correctly called informatics), which should be the field you’re looking at if you want to deeply understand the concept.

Vitor Oct 30, 2022, 6:58 AM
4 points
in reply to: Adam Shai’s comment on: Pondering computation in the real world
Probabilistic Turing machines already exist. They are a standard extension of TMs that transition from state to state with arbitrary probabilities (not just 0 and 1) and can thus easily generate random bits.

Further, we know that deterministic TMs can do anything that probabilistic TMs can, albeit potentially less efficiently.

I suspect you’re not really rejecting TMs per se, but rather the entire theory of computability and complexity that is standard in CS, more specifically the Church-Turing thesis which is the foundation of the whole thing.