My government name is Mack Gallagher. I am a thrift store cashier living in Iowa. Crocker’s Rules. I am an underfunded “alignment” “researcher”. DM me if you’d like to fund my posts.
Try using Marginalia [ I’m unaffiliated ] instead of Google for stuff other than currency conversion and the hours of local businesses.
Thank you for your kind comment! I disagree with the johnswentworth post you linked; it’s misleading to frame NN interpretability as though we started out having any graph with any labels, weird-looking labels or not. I have sent you a DM.
While writing a recent post, I had to decide whether to mention that Nicolaus Bernoulli had written his letter posing the St. Petersburg problem specifically to Pierre Raymond de Montmort, given that my audience and I probably have no other shared semantic anchor for Pierre’s existence, and he doesn’t visibly appear elsewhere in the story.
I decided Yes. I think the idea of awarding credit to otherwise-silent muses in general is interesting.
Footnote to my impending post about the history of value and utility:
After Pascal’s and Fermat’s work on the problem of points, and Huygens’s work on expected value, the next major work on probability was Jakob Bernoulli’s Ars conjectandi, written between 1684 and 1689 and published posthumously by his nephew Nicolaus Bernoulli in 1713. Ars conjectandi had 3 important contributions to probability theory:
[1] The concept that expected experience is conserved, or that probabilities must sum to 1.
Bernoulli generalized Huygens’s principle of expected value in a random event as
[ where is the probability of the th outcome, and is the payout from the th outcome ]
and said that, in every case, the denominator—i.e. the probabilities of all possible events—must sum to 1, because only one thing can happen to you
[ making the expected value formula just
with normalized probabilities! ]
[2] The explicit application of strategies starting with the binomial theorem [ known to ancient mathematicians as the triangle pattern studied by Pascal
and first successfully analyzed algebraically by Newton ] to combinatorics in random games [which could be biased] - resulting in e.g. [ the formula for the number of ways to choose k items of equivalent type, from a lineup of n [unique-identity] items ] [useful for calculating the expected distribution of outcomes in many-turn fair random games, or random games where all more-probable outcomes are modeled as being exactly twice, three times, etc. as probable as some other outcome],
written as :
[ A series of random events [a “stochastic process”] can be viewed as a zig-zaggy path moving down the triangle, with the tiers as events, [whether we just moved LEFT or RIGHT] as the discrete outcome of an event, and the numbers as the relative probability density of our current score, or count of preferred events.
When we calculate , we’re calculating one of those relative probability densities. We’re thinking of as our total long-run number of events, and as our target score, or count of preferred events.
We calculate by first “weighting in” all possible orderings of , by taking , and then by “factoring out” all possible orderings of ways to achieve our chosen W condition [since we always take the same count of W-type outcomes as interchangeable], and “factoring out” all possible orderings of our chosen L condition [since we’re indifferent between those too].
[My explanation here has no particular relation to how Bernoulli reasoned through this.] ]
Bernoulli did not stop with and discrete probability analysis, however; he went on to analyze probabilities [in games with discrete outcomes] as real-valued, resulting in the Bernoulli probability distribution.
[3] The empirical “Law of Large Numbers”, which says that, after you repeat a random game for many turns and add up all the outcomes, the total final outcome will approach the number of turns, times the expected distribution of outcomes in a single turn. E.g. if a die is biased to roll
a 6 40% of the time
a 5 25% of the time
a 4 20% of the time
a 3 8% of the time
a 2 4% of the time, and
a 1 3% of the time
then after 1,000 rolls, your counts should be “close” to
6: .4*1,000 = 400
5: .25*1,000 = 250
4: .2*1,000 = 200
3: .08*1,000 = 80
2: .04*1,000 = 40
1: .03*1,000 = 30
and even “closer” to these ideal ratios after 1,000,000 rolls
- which Bernoulli brought up in the fourth and final section of the book, in the context of analyzing sociological data and policymaking.
One source: “Do Dice Play God?” by Ian Stewart
[ Please DM me if you would like the author of this post to explain this stuff better. I don’t have much idea how clear I am being to a LessWrong audience! ]
This is one of those subject areas that’d be unfortunately bad to get into publicly. If you or any other individual wants to grill me on this, feel free to DM me or contact me by any of the above methods and I will take disclosure case by case.
This is a just ask.
Also, even though it’s not locally rhetorically convenient [ where making an isolated demand for rigor of people making claims like “scaling has hit a wall [therefore AI risk is far]” that are inconvenient for AInotkilleveryoneism, is locally rhetorically convenient for us ], we should demand the same specificity of people who are claiming that “scaling works”, so we end up with a correct world-model and so people who just want to build AGI see that we are fair.
Update: My best current theory [ hasn’t changed in a few months but I figured it might be worth posting ] is that composite smell data [i.e. the better part of smell processing] is passed directly from the olfactory bulb to somewhere in the entorhinal-amygdalar-temporal area, while there are a few scents that function as pheromones in the sense that we have innate responses to the scents as opposed to their associated experiences [ so, skunk and feces as well as the scent of eligible mates ] and data about these scents is relayed by thin, almost invisible projections to the hypothalamus or other nuclei in the “emotional motor system” so the behavioral responses can bootstrap.
What happens at that point depends a lot on the details of the lawbreaker’s creation. [ . . . ] The probability seems unlikely to me to be zero for the sorts of qualities which would make such an AI agent dangerous.
Have you read The Sun is big, but superintelligences will not spare Earth a little sunlight?
I’ll address each of your 4 critiques:
[ 1. ] In public policy making, you have a set of preferences, which you get from votes or surveys, and you formulate policy based on your best objective understanding of cause and effect. The preferences don’t have to be objective, because they are taken as given.
The point I’m making in the post
Well, I reject the presumption of guilt.
is that no matter whether you have to treat the preferences as objective, there is an objective fact of the matter about what someone’s preferences are, in the real world [ real, even if not physical ].
[ 2. ] [ Agreeing on such basic elements of our ontology/epistemology ] isn’t all that relevant to AI safety, because an AI only needs some potentially dangerous capabilities.
Whether or not an AI “only needs some potentially dangerous capabilities” for your local PR purposes, the global truth of the matter is that “randomly-rolled” superintelligences will have convergent instrumental desires that have to do with making use of the resources we are currently using [like the negentropy that would make Earth’s oceans a great sink for 3 x 10^27 joules], but not desires that tightly converge with our terminal desires that make boiling the oceans without evacuating all the humans first a Bad Idea.
[ 3. ] You haven’t defined consciousness and you haven’t explained how [ we can know something that lives in a physical substrate that is unlike ours is conscious ].
My intent is not to say “I/we understand consciousness, therefore we can derive objectively sound-valid-and-therefore-true statements from theories with mentalistic atoms”. The arguments I actually give for why it’s true that we can derive objective abstract facts about the mental world, begin at “So why am I saying this premise is false?”, and end at ”. . . and agree that the results came out favoring one theory or another.” If we can derive objectively true abstract statements about the mental world, the same way we can derive such statements about the physical world [e.g. “the force experienced by a moving charge in a magnetic field is orthogonal both to the direction of the field and to the direction of its motion”] this implies that we can understand consciousness well, whether or not we already do.
[ 4. ] there doesn’t need to be [ some degree of objective truth as to what is valuable ]. You don’t have to solve ethics to set policy.
My point, again, isn’t that there needs to be, for whatever local practical purpose. My point is that there is.
I think, in retrospect, the view that abstract statements about shared non-reductionist reality can be objectively sound-valid-and-therefore true follows pretty naturally from combining the common-on-LessWrong view that logical or abstract physical theories can make sound-valid-and-therefore-true abstract conclusions about Reality, with the view, also common on LessWrong, that we make a lot of decisions by modeling other people as copies of ourselves, instead of as entities primarily obeying reductionist physics.
It’s just that, despite the fact that all the pieces are there, it goes on being a not-obvious way to think, if for years and years you’ve heard about how we can only have objective theories if we can do experiments that are “in the territory” in the sense that they are outside of anyone’s map. [ Contrast with celebrity examples of “shared thought experiments” from which many people drew similar conclusions because they took place in a shared map—Singer’s Drowning Child, the Trolley Problem, Rawls’s Veil of Ignorance, Zeno’s story about Achilles and the Tortoise, Pascal’s Wager, Newcomb’s Problem, Parfit’s Hitchhiker, the St. Petersburg paradox, etc. ]
? Yes, that is the bad post I am rebutting.
Recently, Raginrayguns and Philosophy Bear both [presumably] read “Cargo Cult Science” [not necessarily for the first time] on /r/slatestarcodex. I follow both of them, so I looked into it. And TIL that’s where “cargo-culting” comes from. He doesn’t say why it’s wrong, he just waves his hands and says it doesn’t work and it’s silly. Well, now I feel silly. I’ve been cargo-culting “cargo-culting”. I’m a logical decision theorist. Cargo cults work. If they work unreliably, so do reductionistic methods.
I once thought “slack mattered more than any outcome”. But whose slack? It’s wonderful for all humans to have more slack. But there’s a huge game-theoretic difference between the species being wealthier, and thus wealthier per capita, and being wealthy/high-status/dominant/powerful relative to other people. The first is what I was getting at by “things orthogonal to the lineup”; the second is “the lineup”. Trying to improve your position relative to copies of yourself in a way that is zero-sum is “the rat race”, or “the Red Queen’s race”, where running will ~only ever keep you in the same place, and cause you and your mirror-selves to expend a lot of effort that is useless if you don’t enjoy it.
[I think I enjoy any amount of “the rat race”, which is part of why I find myself doing any of it, even though I can easily imagine tweaking my mind such that I stop doing it and thus exit an LDT negotiation equilibrium where I need to do it all the time. But I only like it so much, and only certain kinds.]
! I’m genuinely impressed if you wrote this post without having a mental frame for the concepts drawn from LDT.
LDT says that, for the purposes of making quasi-Kantian [not really Kantian but that’s the closest thing I can gesture at OTOH that isn’t just “read the Yudkowsky”] correct decisions, you have to treat the hostile telepaths as copies of yourself.
Indexical uncertainty, ie not knowing whether you’re in Omega’s simulation or the real world, means that, even if “I would never do that”, if someone is “doing that” to me, in ways I can’t ignore, I have to act as though I might ever be in a situation where I’m basically forced to “do that”.
I can still preferentially withhold reward from copies of myself that are executing quasi-threats, though. And in fact this is correct because it minimizes quasi-threats in the mutual copies-of-myself negotiating equilibrium.
“Acquire the ability to coerce, rather than being coerced by, other agents in my environment”, is not a solution to anything—because the quasi-Rawlsian [again, not really Rawlsian, but I don’t have any better non-Yudkowsky reference points OTOH] perspective means that if you precommit to acquire power, you end up in expectation getting trodden on just as much as you trod on the other copies of you. So you’re right back where you started.
Basically, you have to control things orthogonal to your position in the lineup, to robustly improve your algorithm for negotiating with others.
And I think “be willing to back deceptions” is in fact such a socially-orthogonal improvement.
Thank you!