testingthewaters

Karma: 378

testingthewaters May 13, 2025, 9:41 PM
16 points
0
on: Too Soon
I’m really sorry for your loss. It’s a crushing thing and as my parents get older I often feel that gnawing terror and anxiety as well. I hope you can find some peace eventually.

testingthewaters May 9, 2025, 4:06 PM
3 points
0
on: An alignment safety case sketch based on debate
I believe that this is the crux of debate:

With computationally unbounded debaters, debate with optimal play (and cross-examination) can answer any question in NEXP given polynomial time judges.

The crux being:

Interesting questions about the world can be formalised as problems with deterministic solutions to be resolved via an iterated proof process.

Yet consider that most “weighty” problems safe AI version N will have to resolve may be of the form:

Forecast the impact of policy X on the world. Is the impact good or bad?

Where X can be “deploy safe AI system version N+1” or “implement governance structure Y” or “release commercial technology patent Z”

In these cases the solution requires some simulation of a computationally irreducibly complex system (the world, which includes safe AI version N). Even assuming no deception, it is very hard to see how the debate process can in anyway approximate a proof of the validity of these propositions, in a mathematical sense.

testingthewaters May 2, 2025, 2:46 PM
3 points
0
on: What are important UI-shaped problems that Lightcone could tackle?
If you want a semi-amibtious idea shaped around UI, how about “serendipity generation for AI Safety/other related topics”

This would be a LW addon or separate site with a few features:
- Showing random extracts of interesting articles/blogposts/papers, curated by humans (perhaps in a “news ticker” style)
- Users could sign up to get matched for one on ones with other users based on relative domain expertise (so matching people who are unlikely to already know one another), for friendly coffee chat or professional networking
- Update board for people to post things they’re working with as a scrolling Twitter like Feed with no algorithm
Designing this well so it isn’t a pain to use would be a fine balance but I think there’s a lot of potential for creating “a big target area for luck” and removing friction between different social cliques in AIS for idea sharing. Also could offer this setup to other fields if you needed revenue, I think physics or CS in general might benefit from such a service.

testingthewaters May 2, 2025, 1:05 AM
5 points
1
on: testingthewaters’s Shortform
Google potentially adding ads to gemini:

https://arstechnica.com/ai/2025/05/google-is-quietly-testing-ads-in-ai-chatbots/

OpenAI adds shopping to chatgpt:

https://www.wired.com/story/openai-adds-shopping-to-chatgpt/

If there’s anything the history of advertising should tell us, it is that there will be powerful optimisation pressures for persuasion being developed quietly in the background for all future model post training pipelines.

testingthewaters Apr 29, 2025, 3:13 PM
1 point
0
in reply to: Ben Pace’s comment on: Wei Dai’s Shortform
Well said. Bravo.

testingthewaters Apr 28, 2025, 11:19 PM
4 points
2
on: testingthewaters’s Shortform
More bad news for optimisation pressures on AI companies: ChatGPT now has a buy product feature

https://www.wired.com/story/openai-adds-shopping-to-chatgpt/

For now they claim that all product recommendations are organic. If you believe this will last I strongly suggest you review the past twenty years of tech company evolution.

testingthewaters Apr 28, 2025, 10:50 PM
2 points
0
in reply to: Wei Dai’s comment on: Wei Dai’s Shortform
Ah yes, but if all these wannabe heroes keep going we’ll be really screwed, so it’s up to me to take a stand against the fools dooming us all… the ratchet of Moloch cranks ever clockwise

testingthewaters Apr 28, 2025, 8:41 AM
2 points
0
in reply to: lillybaeum’s comment on: lillybaeum’s Shortform
It is an extension of the filter bubbles and polarisation issues of the social media era, but yes it is coming into its own as a new and serious threat.

testingthewaters Apr 24, 2025, 2:04 AM
7 points
9
in reply to: Neel Nanda’s comment on: MichaelDickens’s Shortform
At the risk of seeming quite combative, when you say

And I know a lot of safety people at deepmind and other AGI labs who I’m very confident also sincerely care about reducing existential risks. This is one of their primary motivations, they often got into the field due to being convinced by arguments about ai risk, they will often raise in conversation concerns that their current work or the team’s current strategy is not focused on it enough, some are extremely hard-working or admirably willing to forgo credits so long as they think that their work is actually mattering for X-Risk, some dedicate a bunch of time to forming detailed mental models of how AI leads to bad outcomes and how this could be prevented and how their work fit in, etc.

That’s basically what I mean when I said in my comment

AI safety, by its nature, resists the idea of creating powerful new information technologies to exploit mercilessly for revenue without care for downstream consequences. However, many actors in the AI safety movement are themselves tied to the digital economy, and depend on it for their power, status, and livelihoods. Thus, it is not that there are no genuine concerns being expressed, but that at every turn these concerns must be resolved in a way that keeps the massive tech machine going. Those who don’t agree with this approach are efficiently selected against. [examples follow]

And, after thinking about it, I don’t see your statement conflicting with mine.

testingthewaters Apr 24, 2025, 1:47 AM
53 points
13
in reply to: MichaelDickens’s comment on: MichaelDickens’s Shortform
I think this is straightforwardly true and basically hard to dispute in any meaningful way. A lot of this is basically downstream of AI research being part of a massive market/profit generating endeavour (the broader tech industry), which straightforwardly optimises for more and more “capabilities” (of various kinds) in the name of revenue. Indeed, one could argue that long before the current wave of LLMs the tech industry was developing powerful agentic systems that actively worked to subvert human preferences in favour of disempowering them/manipulating them, all in the name of extracting revenue from intelligent work… we just called the AI system the Google/Facebook/Youtube/Twitter Algorithm.

The trend was always clear: an idealistic mission to make good use of global telecommunication/information networks finds initial success and is a good service. Eventually pressures to make profits cause the core service to be degraded in favour of revenue generation (usually ads). Eventually the company accrues enough shaping power to actively reshape the information network in its favour, and begins dragging everything down with it. In the face of this AI/LLMs are just another product to be used as a revenue engine in the digital economy.

AI safety, by its nature, resists the idea of creating powerful new information technologies to exploit mercilessly for revenue without care for downstream consequences. However, many actors in the AI safety movement are themselves tied to the digital economy, and depend on it for their power, status, and livelihoods. Thus, it is not that there are no genuine concerns being expressed, but that at every turn these concerns must be resolved in a way that keeps the massive tech machine going. Those who don’t agree with this approach are efficiently selected against. For example:
- Race dynamics are bad? ~~Maybe we should slow down.~~ We just need to join the race and be the more morally-minded actor. After all, there’s no stopping the race, we’re already locked in.
- Our competitors/other parties are doing dangerous things? ~~Maybe we could coordinate and share our concerns and research with them.~~ We can’t fall behind, we’ve got to fly the AI safety flag at the conference of AI Superpowers. Let’s speed up too.
- New capabilities are unknown and jagged? ~~Let’s just leave well enough alone.~~ Let’s invest more in R&D so we can safely understand and harness them.
- Here’s a new paradigm that might lead to a lot of risk and a lot of reward. ~~We should practice the virtue of silence and buy the world time~~ We should make lots of noise so we can get funding. To study it. Not to use it, of course. Just to understand the safety implications.
- Maybe progress in AI is slower than we thought. ~~Hooray! Maybe we can chill for a bit~~ That’s time for us to exploit our superior AI knowledge and accelerate progress to our benefit.
We’ve seen this before.

To be honest, though, I’m not sure what to do about this. So much has been invested by now that it truly feels like history is moving with a will of its own, rather than individuals steering the ship. Every time I look at what’s going on I feel the sense that maybe I’m just the idiot that hasn’t gotten the signal to hammer that “exploit” button. After all, it’s what everyone else is doing.

A Letter to His Highness Louis XV, the King of France

testingthewatersApr 22, 2025, 12:51 AM

2 points

0 comments1 min readLW link

(aclevername.substack.com)

testingthewaters Apr 19, 2025, 11:35 AM
1 point
0
in reply to: Cole Wyeth’s comment on: A Dissent on Honesty
For my part, I didn’t realise it became so heavily downvoted, but I did not mean it at all in an accusatory or moralizing manner. I also, upon reflection, don’t regret posting it.

testingthewaters Apr 17, 2025, 10:48 AM
−12 points
10
on: A Dissent on Honesty
I’m sorry that your life has come to a point where you might feel like adopting a fake personality and regularly deceiving those around you is how to be happy and get what you want. It’s almost impossible to deliver good advice to a stranger over the internet, but I hope that one day you can find joy and fulfillment in being yourself.

testingthewaters Apr 15, 2025, 4:03 PM
5 points
0
on: OpenAI #13: Altman at TED and OpenAI Cutting Corners on Safety Testing

I think I can indeed forsee the future where OpenAI is helping the Pentagon with its AI weapons. I expect this to happen. I want to be clear that I don’t think this is a bad thing. The risk is in developing highly capable AIs in the first place. As I have said before, Autonomous Killer Robots and AI-assisted weapons in general are not how we lose control over the future to AI, and failing to do so is a key way America can fall behind. It’s not like our rivals are going to hold back. To the extent that the AI weapons scare the hell out of everyone? That’s a feature.

This is giving “guns don’t kill people, people kill people” energy. Sure, ai takeovers can happen without skynet, but skynet sure makes it easier.

testingthewaters Apr 12, 2025, 8:19 PM
6 points
1
on: Why does LW not put much more focus on AI governance and outreach?
The simple answer is related to the population and occupation of the modal lesswrong viewer, and hence the modal lesswrong commenter, and upvoter. The site culture also tends towards skepticism and pessimism of institutions (I do not make a judgement on whether this valence is justified). I however also agree that this is important to at least discuss.

testingthewaters Apr 11, 2025, 2:36 PM
4 points
2
on: testingthewaters’s Shortform
From Inadequate Equilibria:

Visitor: I take it you didn’t have the stern and upright leaders, what we call the Serious People, who could set an example by donning Velcro shoes themselves?

From Ratatouille:

In many ways, the work of a critic is easy. We risk very little, yet enjoy a position over those who offer up their work and their selves to our judgment. We thrive on negative criticism, which is fun to write and to read. But the bitter truth we critics must face, is that in the grand scheme of things, the average piece of junk is probably more meaningful than our criticism designating it so. But there are times when a critic truly risks something, and that is in the discovery and defense of the new. The world is often unkind to new talent, new creations. The new needs friends.

And that’s why bravery is the secret name of the nameless virtue, and seriously underrated.

[[To elaborate slightly: to go beyond pointing and sneering, to actually work to construct a better future, is very difficult. It requires breaking from social conventions, not just the social conventions you claim are “self evidently stupid” but also the ones you see as natural and right. In many ways the hardest task is not to realise what the “right choice” is, but to choose cooperate in face of your knowledge of nash equilibria.

To reach for the pareto optimal solution to a coordination game means knowing you might very well be stabbed in the back. In a world where inadequate equilibria persist the only way we get out is to be the first person to break those equilibria, and that requires you to take some pretty locally irrational actions. Sometimes choosing not to defect or to punish requires unimaginable bravery. Mere knowledge of Moloch does not save you from Moloch, only action does.]]

testingthewaters Apr 5, 2025, 10:27 AM
1 point
0
on: AI CoT Reasoning Is Often Unfaithful
I mean, this applies to humans too. The words and explanations we use for our actions are often just post hoc rationalisations. An efficient text predictor must learn not what the literal words in front of them mean, but the implied scenario and thought process they mask, and that is a strictly nonlinear and “unfaithful” process.

testingthewaters Apr 3, 2025, 12:00 AM
2 points
0
on: testingthewaters’s Shortform
I think I’ve just figured out why decision theories strike me as utterly pointless: they get around the actual hard part of making a decision. In general, decisions are not hard because you are weighing payoffs, but because you are dealing with uncertainty.

To operationalise this: a decision theory usually assumes that you have some number of options, each with some defined payout. Assuming payouts are fixed, all decision theories simply advise you to pick the outcome with the highest utility. “Difficult problems” in decision theory are problems where the payout is determined by some function that contains a contradiction, which is then resolved by causal/evidential/functional decision theories each with their own method of cutting the Gordian knot. The classic contradiction, of course, is that “payout(x1) == 100 iff predictor(your_choice) == x1; else payout(x1) == 1000″.

Except this is not at all what makes real life decisions hard. If I am planning a business and ever get to the point where I know a function for exactly how much money two different business plans will give me, I’ve already gotten past the hard part of making a business plan. Similarly, if I’m choosing between two doors on a game show the difficulty is not that the host is a genius superpredictor who will retrocausally change the posterior goat/car distribution, but the simple fact that I do not know what is behind the doors. Almost all decision theories just skip past the part where you resolve uncertainty and gather information, which makes them effectively worthless in real life. Or, worse, they try to make the uncertainty go away: If I have 100 dollars and can donate to a local homeless shelter I know well or try and give it to a malaria net charity I don’t know a lot about, I can be quite certain the homeless shelter will not misappropriate the funds or mismanage their operation, and less so about the faceless malaria charity. This is entirely missing from the standard EA arguments for allocation of funds. Uncertainty matters.

testingthewaters Apr 1, 2025, 9:03 AM
9 points
0
in reply to: CapResearcher’s comment on: CapResearcher’s Shortform
This has shifted my perceptions of what is in the wild significantly. Thanks for the heads up.

testingthewaters Mar 25, 2025, 8:22 PM
1 point
0
on: testingthewaters’s Shortform
https://research.google/blog/deciphering-language-processing-in-the-human-brain-through-llm-representations/

Activations in LLMs are linearly mappable to activations in the human brain. Imo this is strong evidence for the idea that LLMs/NNs in general acquire extremely human like cognitive patterns, and that the common “shoggoth with a smiley face” meme might just not be accurate

testingthewaters

A Let­ter to His High­ness Louis XV, the King of France

A Letter to His Highness Louis XV, the King of France