Thanks for writing this. I have been fretting for some time and realized that what I needed was a rational take on the war. I appreciate the time you’ve taken you write this out and I’ll check out your other posts on this.
sbenthall
Karma: 367
Reward Hacking from a Causal Perspective
Incentives from a causal perspective
Causality: A Brief Introduction
Introduction to Towards Causal Foundations of Safe AGI
Don’t Fear the Reaper: Refuting Bostrom’s Superintelligence Argument
Autonomy, utility, and desire; against consequentialism in AI design
more on predicting agents
This seems correct to me. Thank you.
You don’t know anything about how cars work?
It’s possible to predict the behavior of black boxes without knowing anything about their internal structure.
Elaborate?
That says a lot more about your personal values then the general human condition.
I suppose you are right.
The models of worms might be a bit better at predicting worm behavior but they are not perfect.
They are significantly closer to being perfect than our models of humans. I think you are right in pointing out that where you draw the line is somewhat arbitrary. But the point is the variation on the continuum.
Do you think it is something external to the birds that make them migrate?
prediction and capacity to represent
AI Tao
Norbert Wiener is where it all starts. This book has a lot of essays. It’s interesting—he’s talking about learning machines before “machine learning” was a household word, but envisioning it as electrical circuits.
http://www.amazon.com/Cybernetics-Second-Edition-Control-Communication/dp/026273009X
I think that it’s important to look inside the boxes. We know a lot about the mathematical limits of boxes which could help us understand whether and how they might go foom.
Thank you for introducing me to that Concrete Mathematics book. That looks cool.
I would be really interested to see how you model this problem. I’m afraid that op-amps are not something I’m familiar with but it sounds like you are onto something.
Do you think that rationalism is becoming a religion, or should become one?
Thanks. That criticism makes sense to me. You put the point very concretely.
What do you think of the use of optimization power in arguments about takeoff speed and x-risk?
Or do you have a different research agenda altogether?
That makes sense. I’m surprised that I haven’t found any explicit reference to that in the literature I’ve been looking at. Is that because it is considered to be implicitly understood?
One way to talk about optimization power, maybe, would be to consider a spectrum between unbounded, LaPlacean rationality and the dumbest things around. There seems to be a move away from this though, because it’s too tied to notions of intelligence and doesn’t look enough at outcomes?
It’s this move that I find confusing.
There are people in my department who do work in this area. I can reach out and ask them.
I think Mechanical Turk gets used a lot for survey experiments because it has a built-in compensation mechanism and there are ways to ask questions in ways that filter people into precisely what you want.
I wouldn’t dismiss Facebook ads so quickly. I bet there is a way to target mobile app developers on that.
My hunch is that like survey questions, sampling methods are going to need to be tuned case-by-case and patterns extracted inductively from that. Good social scientific experiment design is very hard. Standardizing it is a noble but difficult task.
This point about Ukrainian neo-Nazis is very misunderstood by the West.
During the Maidan revolution in Ukraine in 2014, neo-Nazi groups occupied government buildings and brought about a transition of government.
Why are there neo-Nazis in Ukraine? Because during WWII, the Nazis and the USSR were fighting over Ukraine. Ukraine is today quite ethnically diverse, and some of the ‘western’ Ukrainians who were resentful of USSR rule and, later, Russian influence, have reclaimed nazi ideas as part of a far-right Ukrainian nationalism. Some of these nazi groups that were originally militias have been incorporated into the Ukrainian military.
This is all quite well documented:
https://en.wikipedia.org/wiki/2014_Euromaidan_regional_state_administration_occupations
https://jacobin.com/2022/02/maidan-protests-neo-nazis-russia-nato-crimea
One of the regiments most well known to have Nazi ties was defeated at the Siege of Mariupol
https://en.wikipedia.org/wiki/Azov_Regiment
Naturally, this history is downplayed in presentations of Ukrainian nationalism targeted at the West, and emphasized in Russia depictions of Ukraine.