lalaithion

Karma: 648

lalaithion Apr 3, 2023, 5:04 PM
15 points
8
on: Orthogonality is expensive

Orthogonality in design states that we can construct an AGI which optimizes for any goal. Orthogonality at runtime would be an AGI design that would consist of an AGI which can switch between arbitrary goals while operating. Here, we are only really talking about the latter orthogonality

This should not be relegated to a footnote. I’ve always thought that design-time orthogonality is the core of the orthogonality thesis, and I was very confused by this post until I read the footnote.

lalaithion Feb 3, 2023, 8:30 PM
8 points
3
on: Aiming for Convergence Is Like Discouraging Betting
There are tactics I have available to me which are not oriented towards truthseeking, but instead oriented towards “raising my status at the expense of yours”. I would like to not use those tactics, because I think that they destroy the commons. I view “collaborative truth seeking” as a commitment between interlocutors to avoid those tactics which are good at status games or preaching to the choir, and focus on tactics which are good at convincing.
Additionally,
I can can just … listen to the counterarguments and judge them on their merits, without getting distracted by the irrelevancy of whether the person seems “collaborative” with me
I do not have this skill. When I perceive my partner in discourse as non-collaborative, I have a harder time honestly judging their counterarguments, and I have a harder time generating good counterarguments. This means discourse with someone who is not being collaborative takes more effort, and I am less inclined to do it. When I say “this should be a norm in this space”, I am partially saying “it will be easier for you to convince me if you adopt this norm”.

lalaithion Jan 27, 2023, 8:29 PM
5 points
4
in reply to: davidad’s comment on: Assigning Praise and Blame: Decoupling Epistemology and Decision Theory
I walked through some examples of Shapley Value here, and I’m not so sure it satisfies exactly what we want on an object level. I don’t have a great realistic example here, but Shapley Value assigns counterfactual value to individuals who did in fact not contribute at all, if they would have contributed were your higher-performers not present. So you can easily have “dead weight” on a team which has a high Shapley Value, as long as they could provide value if their better teammates were gone.

lalaithion Jan 25, 2023, 8:17 PM
3 points
2
on: Some questions about free will compatibilism
my friend responds, “Your beliefs/desires, and ultimately your choice to study English, was determined by the state of your brain, which was determined by the laws of physics. So don’t feel bad! You were forced to study English by the laws of physics.”
The joke I always told was
“I choose to believe in free will, because if free will does exist, then I’ve chosen correctly, and if free will doesn’t exist, then I was determined to choose this way since the beginning of the universe, and it’s not my fault that I’m wrong!”

lalaithion Jan 25, 2023, 8:14 PM
1 point
0
on: Some questions about free will compatibilism
Reductionism means that “Jones” and “Smith” are not platonic concepts. They’re made out of parts, and you can look at the parts and ask how they contribute to moral responsibility.
When you say “Smith has a brain tumor that made him do it”, you are conceiving of Smith and the brain tumor as different parts, and concluding that the non-tumor part isn’t responsible. If you ask “Is the Smith-and-brain-tumor system responsible for the murder”, the answer is yes. If you break the mind of Jones into parts, you could similarly ask “Is the visual cortex of Jones responsible for the murder”, and the answer would be no.
So, why do we conceive of “Smith” and “the brain tumor” as more naturally separable? Because we have an understanding that the person Smith was before the brain tumor has continuity with the current Smith+brain tumor system, that the brain tumor is separable using surgery, that the post-surgery Smith would have continuity with the current Smith+brain tumor system, that the post-surgery Smith would approve of the removal of the brain tumor, and the pre-tumor Smith would approve of the removal of the brain tumor.
Whereas we don’t have an understanding of Jones in that way. If we did, maybe if we were visited by aliens which had much more advanced neuroscience, and they pointed out that lead exposure had changed a neural circuit so that it operated at 85% efficiency instead of the normal 100%, and they could easily change that, and the post-change Jones didn’t want to commit any murders, we might not consider the new Jones morally responsible. But the old Jones-minus-efficiency was responsible.

lalaithion Nov 10, 2022, 5:17 PM
1 point
0
on: Desiderata for an Adversarial Prior
One adversarial prior would be “my prior for bets of expected value X is O(1/X)”.

lalaithion Nov 6, 2022, 5:21 PM
9 points
2
on: Has anyone increased their AGI timelines?
No public estimates, but the difficulty of self driving cars definitely pushed my AGI timelines back. In 2018 I predicted full self driving by 2023; now that’s looking unlikely. Yes, the advance in text and image understanding and generation has improved a lot since 2018, but instead of shortening my estimates that’s merely rotated which capabilities will come online earlier and which will wait until AGI.

However, I expect some crazy TAI in the next few years. I fully expect “solve all the millennium problems” to be doable without AGI, as well as much of coding/design/engineering work. I also think it’s likely that text models will be able to do the work of a paralegal/research assistant/copywriter without AGI.

lalaithion Oct 27, 2022, 8:38 PM
3 points
3
in reply to: gbear605’s comment on: Is the Orthogonality Thesis true for humans?
Additionally, if you have a problem which can be solved by either (a) crime or (b) doing something complicated to fix it, your ability to do (b) is higher the smarter you are.

lalaithion Oct 10, 2022, 5:12 PM
3 points
0
on: Six (and a half) intuitions for KL divergence
It would be nice to have a couple examples comparing concrete distributions Q and P and examining their KL-divergence, why it’s large or small, and why it’s not symmetric.

lalaithion Oct 7, 2022, 4:48 PM
5 points
0
in reply to: niplav’s comment on: Open & Welcome Thread—Oct 2022
If I were making one up, I might say “g distributes over composition of f”.

lalaithion Sep 26, 2022, 8:53 PM
9 points
4
in reply to: Amelia Bedelia’s comment on: LW Petrov Day 2022 (Monday, 9/26)
It would be better if it was merely an organization that merely had contradictory goals (maybe a degrowth anarcho-socialist group? A hardcore anti-science christian organization?) but wasn’t organized around the dislike of our group specifically.

lalaithion Sep 12, 2022, 5:10 PM
1 point
0
in reply to: nem’s comment on: All AGI safety questions welcome (especially basic ones) [~monthly thread]
More likely, the AI just finds a website with a non-compliant GET request, or a GET request with a SQL injection vulnerability.

lalaithion Aug 28, 2022, 11:20 PM
1 point
0
in reply to: Archimedes’s comment on: What’s the Most Impressive Thing That GPT-4 Could Plausibly Do?
I agree that before that point, an AI will be transformative, but not to the point of “AGI is the world superpower”.

lalaithion Aug 28, 2022, 10:50 PM
4 points
2
in reply to: Archimedes’s comment on: What’s the Most Impressive Thing That GPT-4 Could Plausibly Do?
That’s what people used to say about chess and go. Yes, mathematics requires intuition, but so does chess; the game tree’s too big to be explored fully.

Mathematics requires greater intuition and has a much broader and deeper “game” tree, but once we figure out the analogue to self-play, I think it will quickly surpass human mathematicians.

lalaithion Aug 26, 2022, 9:11 PM
7 points
0
on: What’s the Most Impressive Thing That GPT-4 Could Plausibly Do?
GPT-4 (Edited because I actually realize I put way more than 5% weight on the original phrasing): SOTA on language translation for every language (not just English/French and whatever else GPT-3 has), without fine-tuning.
Not GPT-4 specifically, assuming they keep the focus on next-token prediction of all human text, but “around the time of GPT-4”: Superhuman theorem proving. I expect one of the millennium problems to be solved by an AI sometime in the next 5 years.

lalaithion Jul 15, 2022, 4:31 PM
3 points
0
on: Review of The Engines of Cognition
“metis” is an ancient greek word/goddess which originally meant “magical cunning”, which drifted throughout ancient greek culture to mean something more like “wisdom/prudence/the je ne sais quoi of being able to solve practical problems”.
James C. Scott uses it in his book Seeing Like a State to mean the implicit knowledge passed down through a culture.

lalaithion Jul 2, 2022, 9:00 PM
1 point
0
on: Trends in GPU price-performance
How did you decide where the y-intercept for Huang’s law should be? It seems that even if you fix the slope to 25x per 5 years, the line could still be made to fit the data better by giving it a different y-intercept.

lalaithion Jun 24, 2022, 8:03 PM
3 points
in reply to: tgb’s comment on: Worked Examples of Shapley Values
I think it’s not necessarily the case that free-market pairwise bargaining always leads to the Shapley value. 10Y = X -Y has an infinite number of solutions, and the only principled ways I know of for choosing solutions is either Shapley value or the fact that in this scenario, since there are no other jobs, the owner should be able to negotiate X and Y down to epsilon.

lalaithion Jun 24, 2022, 7:54 PM
1 point
in reply to: Alexander Gietelink Oldenziel’s comment on: Worked Examples of Shapley Values
If you apply Shapley value to a situation where there are more workers than jobs, regardless of how many businesses the jobs are split between, people who can’t get jobs still have a nonzero Shapley value based on their counterfactual contribution to the enterprises they could be working at.

Worked Examples of Shapley Values

lalaithionJun 24, 2022, 5:13 PM

75 points

11 comments8 min readLW link

lalaithion

Worked Ex­am­ples of Shap­ley Values

Worked Examples of Shapley Values