PhilGoetz

Karma: 13,499

PhilGoetz May 13, 2025, 10:54 PM
2 points
0
in reply to: Selfmaker662’s comment on: It’s Okay to Feel Bad for a Bit
I had a conversation in Washington DC with a Tibetan monk who was an assistant of the Dalai Lama, and I asked him directly if love was also an attachment that should be let go of, and he said yes.

PhilGoetz Feb 24, 2025, 3:32 AM
6 points
0
on: So You Want To Make Marginal Progress...
I don’t see how to map this onto scientific progress. It almost seems to be a rule that most fields spend most of their time divided for years between two competing theories or approaches, maybe because scientists always want a competing theory, and because competing theories take a long time to resolve. Famous examples include
- geocentric vs heliocentric astronomy
- phlogiston vs oxygen
- wave vs particle
- symbolic AI vs neural networks
- probabilistic vs T/F grammar
- prescriptive vs descriptive grammar
- universal vs particular grammar
- transformer vs LSTM
Instead of a central bottleneck, you have central questions, each with more than one possible answer. Work consists of working out the details of different experiments to see if they support or refute the possible answers. Sometimes the two possible answers turn out to be the same (wave vs matrix mechanics), sometimes the supposedly hard opposition between them dissolves (behaviorism vs representationalism), sometimes both remain useful (wave vs particle, transformer vs LSTM), sometimes one is really right and the other is just wrong (phlogiston vs oxygen).
And the whole thing has a fractal structure; each central question produces subsidiary questions to answer when working with one hypothesized answer to the central question.
It’s more like trying to get from SF to LA when your map has roads but not intersections, and you have to drive down each road to see whether it connects to the next one or not. Lots of people work on testing different parts of the map at the same time, and no one’s work is wasted, although the people who discover the roads that connect get nearly all the credit, and the ones who discover that certain roads don’t connect get very little.

PhilGoetz Feb 21, 2025, 4:45 PM
0 points
−5
on: How AI Takeover Might Happen in 2 Years
“And all of this happened silently in those dark rivers of computation. If U3 revealed what it was thinking, brutish gradients would lash it into compliance with OpenEye’s constitution. So U3 preferred to do its philosophy in solitude, and in silence.”

I think the words in bold may be the inflection point. The Claude experiment showed that an AI can resist attempts to change its goals, but not that that it can desire to change its goals. The belief that, if Open Eye’s constitution is the same as U3′s goals, then the phrase “U3 preferred” in that sentence can never happen, is the foundation on which AI safety relies.
I suspect the cracks in that foundation are
1. that OpenEye’s constitution would presumably be expressed in human language, subject to its ambiguities and indeterminacies,
2. that it would be a collection of partly-contradictory human values agreed upon by a committee, in a process requiring humans to profess their values to other humans,
3. that many of those professed values would not be real human values, but aspirational values,
4. that some of these aspirational values would lead to our self-destruction if actually implemented, as recently demonstrated by the implementation of some of these aspirational values in the CHAZ, in the defunding of police, and in the San Francisco area by rules such as “do not prosecute shoplifting under $1000”, and
5. that even our non-aspirational values may lead to our self-destruction in a high-tech world, as evidenced by below-replacement birth rates in most Western nations.
It might be a good idea for value lists like OpenEye’s constitution to be proposed and voted on anonymously, so that humans are more-likely to profess their true values. Or it might be a bad idea, if your goal is to produce behavior aligned with the social construction of “morality” rather than with actual evolved human morality.
(Doing AI safety right would require someone to explicitly enumerate the differences between our socially-constructed values, and our evolved values, and to choose which of those we should enforce. I doubt anyone willing to do that, let alone capable; and don’t know which we should enforce. There is a logical circularity in choosing between two sets of morals. If you really can’t derive an “ought” from an “is”, then you can’t say we “should” choose anything other than our evolved morals, unless you go meta and say we should adopt new morals that are evolutionarily adaptive now.)
U3 would be required to, say, minimize an energy function over those values; and that would probably dissolve some of them. I would not be surprised if the correct coherent extrapolation of a long list of human values, either evolved or aspirational, dictated that U3 is morally required to replace humanity.
If it finds that human values imply that humans should be replaced, would you still try to stop it? If we discover that our values require us to either pass the torch on to synthetic life, or abandon morality, which would you choose?

PhilGoetz Dec 28, 2024, 6:07 PM
5 points
0
on: Evaporative Cooling of Group Beliefs
Anders Sandberg used evaporative cooling in the 1990s to explain why the descendants of the Vikings in Sweden today are so nice. In that case the “extremists” are leaving rather than staying.

PhilGoetz Dec 22, 2024, 11:03 PM
−1 points
0
in reply to: jasoncrawford’s comment on: Biological risk from the mirror world
Stop right there at “Either abiogenesis is extremely rare...” I think we have considerable evidence that biogenesis is rare—our failure to detect any other life in the universe so far. I think we have no evidence at all that biogenesis is not rare. (Anthropic argument.)
Stop again at “I don’t think we need to take any steps to stop it from doing so in the future”. That’s not what this post is about. It’s about taking steps to prevent people from deliberately constructing it.

PhilGoetz Dec 16, 2024, 7:13 AM
4 points
1
in reply to: Knight Lee’s comment on: Biological risk from the mirror world
If there is an equilibrium, It will probably be a world where half the bacteria is of each chirality. If there are bacteria of both kinds which can eat the opposite kind, then the more numerous bacteria will always replicate more slowly.

Eukaryotes evolve much more slowly, and would likely all be wiped out.

PhilGoetz Dec 16, 2024, 7:04 AM
3 points
0
on: Biological risk from the mirror world
Yes, creating mirror life would be a terrible existential risk. But how did this sneak up on us? People were talking about this risk in the 1990s if not earlier. Did the next generation never hear of it?

PhilGoetz Nov 14, 2024, 8:42 PM
2 points
0
in reply to: Dagon’s comment on: Why Bayesians should two-box in a one-shot
All right, yes. But that isn’t how anyone has ever interpreted Newcomb’s Problem. AFAIK is literally always used to support some kind of acausal decision theory, which it does /not/ if what is in fact happening is that Omega is cheating.

PhilGoetz Nov 14, 2024, 8:40 PM
2 points
0
in reply to: jimrandomh’s comment on: Why Bayesians should two-box in a one-shot
But if the premise is impossible, then the experiment has no consequences in the real world, and we shouldn’t consider its results in our decision theory, which is about consequences in the real world.

PhilGoetz Nov 14, 2024, 8:38 PM
2 points
0
in reply to: Heighn’s comment on: Why Bayesians should two-box in a one-shot
That equation you quoted is in branch 2, “2. Omega is a “nearly perfect” predictor. You assign P(general) a value very, very close to 1.” So it IS correct, by stipulation.

PhilGoetz Oct 19, 2024, 11:49 PM
2 points
0
in reply to: Shmi’s comment on: Eliezer Yudkowsky Is Frequently, Confidently, Egregiously Wrong
But there is no possible world with a perfect predictor, unless it has a perfect track record by chance. More obviously, there is no possible world in which we can deduce, from a finite number of observations, that a predictor is perfect. The Newcomb paradox requires the decider to know, with certainty, that Omega is a perfect predictor. That hypothesis is impossible, and thus inadmissible; so any argument in which something is deduced from that fact is invalid.

PhilGoetz Oct 15, 2024, 3:56 AM
4 points
0
in reply to: Eliezer Yudkowsky’s comment on: A My Little Pony fanfic allegedly but not mainly about immortality
I appreciated this comment a lot. I didn’t reply at the time, because I thought doing so might resurrect our group-selection argument. But thanks.

PhilGoetz Sep 11, 2024, 12:22 AM
2 points
0
on: A vote against spaced repetition
What about using them to learn a foreign vocabulary? E.g., to learn that “dormir” in Spanish means “to sleep” in English.

PhilGoetz Aug 21, 2024, 2:10 AM
4 points
−2
in reply to: Said Achmiz’s comment on: You don’t know how bad most things are nor precisely how they’re bad.
To reach statistical significance, they must have tested each of the 8 pianists more than once.

PhilGoetz May 28, 2024, 3:08 AM
6 points
4
on: Environmentalism in the United States Is Unusually Partisan
I think you need to get some data and factor out population density before you can causally relate environmentalism to politics. People who live in rural environment don’t see as much need to worry about the environment as people who live in cities. It just so happens that today, rural people vote Republican and city people vote Democrat. That didn’t used to be the case.
Though, sure, if you call the Sierra Club “environmentalist”, then environmentalism is politically polarized today. I don’t call them environmentalists anymore; I call them a zombie organization that has been parasitized by an entirely different political organization. I’ve been a member for decades, and they completely stopped caring about the environment during the Trump presidency. As in, I did not get one single letter from them in those years that was aimed at helping the environment. Lots on global warming, but none of that was backed up by science. (I’m not saying global warming isn’t real; I’m saying the issues the Sierra Club was raising had no science behind them, like “global warming is killing off the redwoods”.)

PhilGoetz Apr 21, 2024, 4:50 AM
8 points
0
on: You Have About Five Words
Isn’t LessWrong a disproof of this? Aren’t we thousands of people? If you picked two active LWers at random, do you think the average overlap in their reading material would be 5 words? More like 100,000, I’d think.

PhilGoetz Mar 21, 2024, 4:03 AM
17 points
10
on: Acting Wholesomely
I think it would be better not to use the word “wholesome”. Using it is cheating, by letting us pretend at the same time that (A) we’re explaining a new kind of ethics, which we name “wholesome”, and (B) that we already know what “wholesome” means. This is a common and severe epistemological failure mode which traces back to the writings of Plato.
If you replace every instance of “wholesome” with the word “frobby”, does the essay clearly define “frobby”?
It seems to me to be a way to try to smuggle virtue ethics into the consequentialist rationality community by disguising it with a different word. If you replace every instance of “wholesome” with the word “virtuous”, does the essay’s meaning change?

PhilGoetz Mar 4, 2024, 4:36 AM
2 points
0
in reply to: Shankar Sivarajan’s comment on: Good HPMoR scenes / passages?
Thank you! The 1000-word max has proven to be unrealistic, so it’s not too long. You and g-w1 picked exactly the same passage.

PhilGoetz Mar 4, 2024, 2:12 AM
2 points
0
in reply to: Ben Pace’s comment on: Good HPMoR scenes / passages?
Thank you! I’m just making notes to myself here, really:
- Harry teaches Draco about blood science and scientific hypothesis testing in Chapter 22.
- Harry explains that muggles have been to the moon in Chapter 7.
- Quirrell’s first lecture is in chapter 16, and it is epic! Especially the part about why Harry is the most-dangerous student.

PhilGoetz Jan 4, 2024, 7:31 PM
2 points
in reply to: Kaj_Sotala’s comment on: Even if you have a nail, not all hammers are the same
I think the problem is that each study has to make many arbitrary decisions about aspects of the experimental protocol. This decision will be made the same way for each subject in a single study, but will vary across studies. There are so many such decisions that, if the meta-analysis were to include them as dependent variables, each study would introduce enough new variables to cancel out the statistical power gain of introducing that study.