Co-Founder & Director—AI Safety ANZ (join us: www.aisafetyanz.com.au)
Advisory Board Member (Growth) - Giving What We Can
Creating Superintelligent Artificial Agents (SAA) without a worldwide referendum is ethically unjustifiable. Until a consensus is reached on whether to bring into existence such technology, a global moratorium is required (we already have AGI).
What is your AI Capabilities Red Line Personal Statement? It should read something like “when AI can do X in Y way, then I think we should be extremely worried / advocate for a Pause*”.
I think it would be valuable if people started doing this; we can’t feel when they’re on an exponential, so its likely we will have powerful AI creep up on us.
@Greg_Colbourn just posted this and I have an intuition that people are going to read it and say “while it can do Y it still can’t do X”
*in the case you think a Pause is ever optimal.
I’d like to see research that uses Jonathan Haidt’s Moral Foundations research and the AI risk repository to forecast whether there will soon be a moral backlash / panic against frontier AI models.
Ideas below from Claude:
# Conservative Moral Concerns About Frontier AI Through Haidt’s Moral Foundations
## Authority/Respect Foundation
### Immediate Concerns
- AI systems challenging traditional hierarchies of expertise and authority
- Undermining of traditional gatekeepers in media, education, and professional fields
- Potential for AI to make decisions traditionally reserved for human authority figures
### Examples
- AI writing systems replacing human editors and teachers
- AI medical diagnosis systems challenging doctor authority
- Language models providing alternative interpretations of religious texts
## Sanctity/Purity Foundation
### Immediate Concerns
- AI “thinking” about sacred topics or generating religious content
- Artificial beings engaging with spirituality and consciousness
- Degradation of “natural” human processes and relationships
### Examples
- AI spiritual advisors or prayer companions
- Synthetic media creating “fake” human experiences
- AI-generated art mixing sacred and profane elements
## Loyalty/Ingroup Foundation
### Immediate Concerns
- AI systems potentially eroding national sovereignty
- Weakening of traditional community bonds
- Displacement of local customs and practices
### Examples
- Global AI systems superseding national control
- AI replacing local business relationships
- Automated systems reducing human-to-human community interactions
## Fairness/Reciprocity Foundation
### Immediate Concerns
- Unequal access to AI capabilities creating new social divides
- AI systems making decisions without clear accountability
- Displacement of workers without reciprocal benefits
### Examples
- Elite access to powerful AI tools
- Automated systems making hiring/firing decisions
- AI wealth concentration without clear social benefit
## Harm/Care Foundation
### Immediate Concerns
- Potential for AI systems to cause unintended societal damage
- Psychological impact on children and vulnerable populations
- Loss of human agency and autonomy
### Examples
- AI manipulation of emotions and behavior
- Impact on childhood development
- Dependency on AI systems for critical decisions
## Key Trigger Points for Conservative Moral Panic
1. Religious Content Generation
- AI systems creating or interpreting religious texts
- Automated spiritual guidance
- Virtual religious experiences
2. Traditional Value Systems
- AI challenging traditional moral authorities
- Automated ethical decision-making
- Replacement of human judgment in moral domains
3. Community and Family
- AI impact on child-rearing and education
- Disruption of traditional social structures
- Replacement of human relationships
4. National Identity
- Foreign AI influence on domestic affairs
- Cultural homogenization through AI
- Loss of local control over technology
## Potential Flashpoints for Immediate Backlash
1. Education System Integration
- AI teachers and tutors
- Automated grading and assessment
- Curriculum generation
2. Religious Institution Interaction
- AI pastoral care
- Religious text interpretation
- Spiritual counseling
3. Media and Information Control
- AI content moderation
- Synthetic media generation
- News creation and curation
4. Family and Child Protection
- AI childcare applications
- Educational AI exposure
- Family privacy concerns
I think there is a 10-20 per cent chance we get digital agents in 2025 that produce a holy shit moment as big as the launch of chatgpt.
If that happens I think that will produce another round of questions that sounds approximately like “how were we so unprepared for this moment”.
Fool me once, shame on you…
I am 90% sure that most AI Safety talent aren’t thinking hard enough about what Neglectedness. The industry is so nascent that you could look at 10 analogous industries, see what processes or institutions are valuable and missing and build an organisation around the highest impact one.
The highest impact job ≠ the highest impact opportunity for you!
AI Safety (in the broadest possible sense, i.e. including ethics & bias) is going be taken very seriously soon by Government decision makers in many countries. But without high quality talent staying in their home countries (i.e. not moving to UK or US), there is a reasonable chance that x/c-risk won’t be considered problems worth trying to solve. X/c-risk sympathisers need seats at the table. IMO local AIS movement builders should be thinking hard about how to keep talent local (this is an extremely hard problem to solve).
Big AIS news imo: “The initial members of the International Network of AI Safety Institutes are Australia, Canada, the European Union, France, Japan, Kenya, the Republic of Korea, Singapore, the United Kingdom, and the United States.”
H/T @shakeel
I beta tested a new movement building format last night: online networking. It seems to have legs.
V quick theory of change:
> problem it solves: not enough people in AIS across Australia (especially) and New Zealand are meeting each other (this is bad for the movement and people’s impact).
> we need to brute force serendipity to create collabs.
> this initiative has v low cost
quantitative results:
> I purposefully didn’t market it hard because it was a beta. I literally got more people that I hoped for
> 22 RSVPs and 18 attendees
> this says to me I could easily get 40+
> average score for below question was 7.27, which is very good for a beta test
I used Zoom, which was extremely clunky. These results suggest to me I should;
> invest in software designed for this use case, not zoom
> segment by career stream (governance vs technical) and/or experience (beginner vs advanced)
> run it every second month
I have heaps of qualitative feedback from participants but don’t have time to share it here.
Email me if interested: yanni@aisafetyanz.com.au
Is anyone in the AI Governance-Comms space working on what public outreach should look like if lots of jobs start getting automated in < 3 years?
I point to Travel Agents a lot not to pick on them, but because they’re salient and there are lots of them. I think there is a reasonable chance in 3 years that industry loses 50% of its workers (3 million globally).
People are going to start freaking out about this. Which means we’re in “December 2019” all over again, and we all remember how bad Government Comms were during COVID.
Now is the time to start working on the messaging!
Three million people are employed by the travel (agent) industry worldwide. I am struggling to see how we don’t lose 80%+ of those jobs to AI Agents in 3 years (this is ofc just one example). This is going to be an extremely painful process for a lot of people.
I should have been clearer—I like the fact that they’re trying to create a larger tent and (presumably) win ethics people over. There are many reasons not also not like the ad. I would also guess that they have an automated campaign running with several (maybe dozens) of pieces of creative. Without seeing their analytics it would be impossible to know which performs the best, but it wouldn’t surprise me if it was this one (lists work).
I really like this ad strategy from BlueDot. 5 stars.
90% of the message is still x / c-risk focussed, but by including “discrimination” and “loss of social connection” they’re clearly trying to either (or both);
create a big tent
nudge people that are in “ethics” but sympathetic to x / c-risk into x / c-risk
(prediction: 4000 disagreement Karma)
Thanks for the heads up! Should be there now :)
[IMAGE] Extremely proud and excited to watch Greg Sadler (CEO of Good Ancestors Policy) and Soroush Pour (Co-Founder of Harmony Intelligence) speak to the Australian Federal Senate Select Committee on Artificial Intelligence. This is a big moment for the local movement. Also, check out who they ran after.
I think how delicately you treat your personal Overton Window should also depend on your timelines.
My instinct as to why people don’t find it a compelling argument;
They don’t have short timelines like me, and therefore chuck it out completely
Are struggling to imagine a hostile public response to 15% unemployment rates
Copium
I always get downvoted when I suggest that (1) if you have short timelines and (2) you decide to work at a Lab then (3) people might hate you soon and your employment prospects could be damaged.
What is something obvious I’m missing here?
One thing I won’t find convincing is someone pointing at the Finance industry post GFC as a comparison.
I believe the scale of unemployment could be much higher. E.g. 5% ->15% unemployment in 3 years.
Thanks for the feedback! I had a feeling this is where I’d land :|
I’d like to quickly skill up on ‘working with people that have a kind of neurodivergence’.
This means;
understanding their unique challenges
how I can help them get the best from/for themselves
but I don’t want to sink more than two hours into this
Can anyone recommend some online content for me please?
I spent 8 years working in strategy departments for Ad Agencies. If you’re interested in the science behind brand tracking, I recommend you check out the Ehrenberg-Bass Institutes work on Category Entry Points: https://marketingscience.info/research-services/identifying-and-prioritising-category-entry-points/