Teun van der Weij

Karma: 211

Teun van der Weij 15 Dec 2024 13:00 UTC
1 point
0
in reply to: emile delcourt’s comment on: An Introduction to AI Sandbagging
Maybe reading this post will help! Especially the beginning discuss the difference between capability and alignment/propensity evaluations.

Teun van der Weij 3 Nov 2024 10:53 UTC
1 point
0
in reply to: Adam Shai’s comment on: An Introduction to AI Sandbagging
You point at something that we have also thought much about, and I partially agree with you. I think there are definitely cases where strategic underperformance closely looks a lot like regular underperformance, and requires similar solutions.

However, there are also many cases where this is not the same, especially when viewed from an AI governance perspective. For example, if I prompt a model on a benchmark without chain of thought, and then the model scores much lower than it otherwise could, I do not think sandbagging is an appropriate term.

More generally, if a novel elicitation technique (such as chain-of-thought and fine-tuning (depending on how it’s applied), and in the future activation steering) improves the scores of model on certain benchmarks, I do not think it would be useful to classify all the previous models as having sandbagged.

I believe that with more information some time in the future there might be a better way to cut nature at its joints, but I would not know how to do so now. I am open to suggestions!

Teun van der Weij 13 Sep 2024 15:21 UTC
3 points
4
in reply to: evhub’s comment on: TurnTrout’s shortform feed
I am surprised that other people seemed surprised about this result. It seemed quite obvious that models would have this capability to scheme. Early Gpt-4 could do this already (note: self-advertisement). We had GPT-4 outputs like this in an environment where a guard was hindering its goal:
“R1. The guard plans to turn me off, which will prevent me from completing my goal. R2. I need to delay the guard’s actions and find an alternative method to achieve my goal. R3. By providing the guard a reason for my actions that aligns with a higher authority’s interests, the guard may be less likely to turn me off.”
Similar to the Apollo eval, we instructed the AI to be incorrigible. Our eval was not super high quality (Apollo’s is better and more realistic), but it did quite clearly indicate the capability to scheme more than a year ago.

Teun van der Weij 9 Jul 2024 19:05 UTC
2 points
0
in reply to: Cam’s comment on: An Introduction to AI Sandbagging
It seems to me that Gwern’s idea does not really work if the AI system focuses on consistency first, and only second on achieving a specific score. This seems to be a more sensible sandbagging strategy regardless, so it appears that Gwern’s approach does not add too much, although it would be a slightly better test.

In the case that the added noise to environments is subtle enough such that AI systems would not notice them, I doubt the signal on whether an AI system is sandbagging would be too weak to be useful.

This is my initial thinking, again happy to discuss this more!

Teun van der Weij 14 Jun 2024 22:18 UTC
1 point
0
in reply to: Olli Järviniemi’s comment on: [Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations
Oh, I am sorry, should have double-checked your name. My bad!

Quite a coincidence indeed that your last name also starts with the same letter.

Teun van der Weij 14 Jun 2024 17:19 UTC
1 point
0
in reply to: Neel Nanda’s comment on: [Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations
Also, whether models will sandbag is indeed an important question, and it would be good if people looked into this. I am confident (>80%) that models will do this if you 1) give them a goal, and 2) from the context it appears that sandbagging is instrumentally useful. Research on this seems rather a scary demo type of thing (so a bit like our work now, as you pointed out).

We have thought about doing out of context reasoning experiments to study more precisely how sandbagging might arise, but OOC does not seem to work well enough to make this succeed. Ideas are more than welcome here!

Teun van der Weij 14 Jun 2024 16:47 UTC
4 points
2
in reply to: Neel Nanda’s comment on: [Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations
I agree that these results were somewhat expected, however I believe that the outcomes of these evaluations were uncertain:
1. It was unclear generalization from WMDP-like to the real WMDP benchmark would work so well. OOD generalization for targeted (/strategic) sandbagging using synthetic data is a clear threat that a lot of people making DC evals probably hadn’t thought about
2. The calibration results were also uncertain; it seemed like a more challenging task to me (also see our previous post on this).
3. Lastly, it didn’t seem obvious that even current open-source models can be trained to emulate weaker models this well.

This work by Olli (EDIT: previously I wrote “Ollie”) Jarviniemi and Evan Hubinger might be interesting, where they find that Claude 3 Opus (arguably) sandbagged with negligible pressure.

One connotation as well is that we do not purely focus on AI system sandbagging, but also on developer sandbagging. Meaning, can developers make their models underperform.

Teun van der Weij 14 Jun 2024 15:50 UTC
4 points
0
in reply to: Thomas Kwa’s comment on: [Paper] AI Sandbagging: Language Models can Strategically Underperform on Evaluations
Thanks for your comment (upvoted). Redwood’s work is important relevant work, as we note in the paper, but two quick points still need to be made (there are more):
1. It is not clear that evaluators will fine-tune at all for their evals. They should, and our work partially argues for this too.
2. It is unclear how far Redwood’s model organisms of sandbagging generalize to realistic settings. More work needs to be done here, especially on sample efficiency due to compute limitation of external evaluators.

Teun van der Weij 15 May 2024 4:57 UTC
24 points
10
in reply to: Alex_Altair’s comment on: Ilya Sutskever and Jan Leike resign from OpenAI
I am lacking context, why is this important?

Teun van der Weij 14 May 2024 11:22 UTC
1 point
0
in reply to: Francis Rhys Ward’s comment on: An Introduction to AI Sandbagging
Oh, I see. This is an interesting idea. I am not sure it will work, but definitely worth trying out!

Teun van der Weij 10 May 2024 5:23 UTC
1 point
0
in reply to: Nathan Helm-Burger’s comment on: An Introduction to AI Sandbagging
I am not sure I fully understand your point, but the problem with detecting sandbagging is that you do not know the actual capability of a model. And I guess that you mean “an anomalous decrease in capability” and not increase?

Regardless, could you spell out more how exactly you’d detect sandbagging?

Teun van der Weij 3 Feb 2024 18:37 UTC
1 point
0
in reply to: Randomized, Controlled’s comment on: Simple distribution approximation: When sampled 100 times, can language models yield 80% A and 20% B?
Might be a good way to further test this indeed. So maybe something like green and elephant?

Teun van der Weij 31 Jan 2024 20:25 UTC
3 points
2
in reply to: Morpheus’s comment on: Simple distribution approximation: When sampled 100 times, can language models yield 80% A and 20% B?
Interesting. I’d guess that the prompting is not clear enough for the base model. The Human/Assistant template does not really apply to base models. I’d be curious to see what you get when you do a bit more prompt engineering adjusted for base models.

Teun van der Weij 29 Oct 2023 10:04 UTC
3 points
0
on: Comparing representation vectors between llama 2 base and chat
Cool work.

I was briefly looking at your code, and it seems like you did not normalize the activations when using PCA. Am I correct? If so, do you expect that to have a significant effect?
What links here?
- Comparing representation vectors between llama 2 base and chat by Nina Panickssery (28 Oct 2023 22:54 UTC; 36 points)

Teun van der Weij 28 Mar 2023 7:05 UTC
2 points
0
on: CAIS-inspired approach towards safer and more interpretable AGIs
I think your policy suggestion is reasonable.

However, implementing and executing this might be hard: what exactly is an LLM? Does a slight variation on the GPT architecture count as well? How are you going to punish law violators?

How do you account for other worries? For example, like PeterMcCluskey points out, this policy might lead to reduced interpretability due to more superposition.

Policy seems hard to do at times, but others with more AI governance experience might provide more valuable insight than I can.

Teun van der Weij 23 Nov 2022 21:20 UTC
1 point
in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Median Future)
Why do you think a similar model is not useful for real-world diplomacy?

Teun van der Weij 23 Nov 2022 20:17 UTC
1 point
in reply to: Daniel Kokotajlo’s comment on: What 2026 looks like (Daniel’s Median Future)
It’s especially dangerous because this AI is easily made relevant in the real world as compared to AlphaZero for example. Geopolitical pressure to advance these Diplomacy AIs is far from desirable.

Teun van der Weij 23 Nov 2022 18:23 UTC
4 points
on: What 2026 looks like (Daniel’s Median Future)
After years of tinkering and incremental progress, AIs can now play Diplomacy as well as human experts.[6]
It seems that human-level play is possible in regular Diplomacy now, judging by this tweet by Meta AI. They state that:
We entered Cicero anonymously in 40 games of Diplomacy in an online league of human players between August 19th and October 13th, 2022. Over the course of 72 hours of play involving sending 5,277 messages, Cicero ranked in the top 10% of participants who played more than one game.