hairyfigment

Karma: 2,077

hairyfigment May 1, 2023, 10:26 PM
−9 points
−4
in reply to: paulfchristiano’s comment on: The Rocket Alignment Problem, Part 2
Even focusing on that doesn’t make your claim appear sensible, because such laws will neither happen soon enough, nor in a sufficiently well-aimed fashion, without work from people like the speaker. You also implied twice that tech CEOs would take action on their own—the quote is in the grandparent—and in the parent you act like you didn’t make that bizarre claim.

hairyfigment May 1, 2023, 8:38 PM
−4 points
−13
in reply to: paulfchristiano’s comment on: The Rocket Alignment Problem, Part 2
>Instead it just means that Bob shouldn’t rely on his company doing the fastest and easiest thing and having it turn out fine. Instead Bob should expect to make sacrifices, either burning down a technical lead or operating in (or helping create) a regulatory environment where the fastest and easiest option isn’t allowed.
The above feels so bizarre that I wonder if you’re trying to reach Elon Musk personally. If so, just reach out to him. If we assume there’s no self-reference paradox involved, we can safely reject your proposed alternatives as obviously impossible; they would have zero credibility even if AI companies weren’t in an arms race, which appears impossible to stop from the inside unless all the CEOs involved can meet at Bohemian Grove.

hairyfigment Apr 27, 2023, 4:19 AM
0 points
−2
in reply to: Lao Mein’s comment on: My Assessment of the Chinese AI Safety Community
See, that makes it sound like my initial response to the OP was basically right, and you don’t understand the argument being made here. At least one Western reading of these new guidelines was that, if they meant anything, then the bureaucratic obstacle they posed for AGI would greatly reduce the threat thereof. This wouldn’t matter if people were happy to show initiative—but if everyone involved thinks volunteering is stupid, then whose job is it to make sure the official rules against a competitive AI project won’t stop it from going forward? What does that person reliably get for doing the job?

hairyfigment Apr 26, 2023, 6:48 PM
3 points
0
in reply to: Lao Mein’s comment on: My Assessment of the Chinese AI Safety Community
All of that makes sense except the inclusion of “EA,” which sounds backwards. I highly doubt Chinese people object to the idea of doing good for the community, so why would they object to helping people do more good, according to our best knowledge?
What links here?
- yungleanbonglord's comment on AI Safety in China: Part 2 by Lao Mein (May 23, 2023, 12:53 PM; 9 points)

hairyfigment Apr 24, 2023, 2:07 AM
7 points
0
in reply to: faul_sname’s comment on: Contra Yudkowsky on AI Doom
I note in passing that the elephant brain is not only much larger, but also has many more neurons than any human brain. Since I’ve no reason to believe the elephant brain is maximally efficient, making the same claim for our brains should require much more evidence than I’m seeing.

hairyfigment Apr 23, 2023, 8:34 PM
14 points
6
on: Could a superintelligence deduce general relativity from a falling apple? An investigation
What are you trying to argue for? I’m getting stuck on the oversimplified interpretation you give for the quote. In the real world, smart people such as Leibniz raised objections to Newton’s mechanics at the time, objections which sound vaguely Einsteinian and not dependent on lots of data. The “principle of sufficient reason” is about internal properties of the theory, similar to Einstein’s argument for each theory of relativity. (Leibniz’s argument could also be given a more Bayesian formulation, saying that if absolute position in space is meaningful, then a full description of the ‘initial state’ of the universe needs to contain additional complexity which has zero predictive value in order to specify that location.) Einstein, in the real world, expressed confidence in general relativity prior to experimental confirmation. What Eliezer is talking about seems different in degree, but not in kind.

hairyfigment Apr 8, 2023, 10:12 AM
3 points
0
in reply to: Raemon’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down
Out of curiosity, what do you plan to do when people keep bringing up Penrose?

hairyfigment Apr 3, 2023, 11:05 PM
2 points
0
in reply to: DirectedEvolution’s comment on: Monthly Roundup #5: April 2023
Pretty sure that doesn’t begin to address the reasons why a paranoid dictator might invade Taiwan, and indeed would undo a lot of hard work spent signaling that the US would defend Taiwan without committing us to nuclear war.

hairyfigment Mar 31, 2023, 10:00 PM
1 point
−1
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
Pretty sure this is my last comment, because what you just quoted about soundness is, in fact, a direct consequence of Löb’s Theorem. For any proposition P, Löb’s Theorem says that □(□P→P)→□P. Let P be a statement already disproven, e.g. “2+2=5”. This means we already had □¬P, and now we have □(¬P & P), which is what inconsistency means. Again, it seemed like you understood this earlier.

hairyfigment Mar 31, 2023, 7:55 PM
2 points
0
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
https://en.wikipedia.org/wiki/Tarski%27s_undefinability_theorem
A coherent formal system can’t fully define truth for its own language. It can give more limited definitions for the truth of some statement, but often this is best accomplished by just repeating the statement in question. (That idea is also due to Tarski: ‘snow is white’ is true if and only if snow is white.) You could loosely say (very loosely!) that a claim, in order to mean anything, needs to point to its own definition of what it would mean for that claim to be true. Any more general definition of truth, for a given language, needs to appeal to concepts which can’t be expressed within that language, in order to avoid a self-reference paradox.
So, there’s no comparison between applying the concept in your last paragraph to individual theorems you’ve already proven, like “2+2=4”—“my system thinks it has proved it, and believes that’s good enough to act on”, and the application to all hypothetical theorems you might prove later, like “2+2=5”. Those two ideas have different meanings—the latter can’t even be expressed within the language in a finite way, though it could be an infinite series of theorems or new axioms of the form □P→P—and they have wildly different consequences. You seem to get this when it comes to hypothetical proofs that eating babies is mandatory.

hairyfigment Mar 31, 2023, 1:04 AM
7 points
0
in reply to: gwd’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
Here’s some more:
A majority (55%) of Americans are now worried at least somewhat that artificially intelligent machines could one day pose a risk to the human race’s existence. This marks a reversal from Monmouth’s 2015 poll, when a smaller number (44%) was worried and a majority (55%) was not.
https://www.monmouth.edu/polling-institute/reports/monmouthpoll_us_021523/

hairyfigment Mar 31, 2023, 12:53 AM
2 points
0
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
The first part of the parent comment is perfectly true for a specific statement—obviously not for all P—and in fact this was the initial idea which inspired the theorem. (For the normal encoding, “This statement is provable within PA,” is in fact true for this exact reason.) The rest of your comment suggests you need to more carefully distinguish between a few concepts:
1. what PA actually proves
2. what PA would prove if it assumed □P→P for all claims P
3. what is actually true, which (we believe) includes category 1 but emphatically not 2.

hairyfigment Mar 31, 2023, 12:28 AM
2 points
0
in reply to: gwd’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
This may be what I was thinking of, though the data is more ambiguous or self-contradictory: https://www.vox.com/future-perfect/2019/1/9/18174081/fhi-govai-ai-safety-american-public-worried-ai-catastrophe

hairyfigment Mar 30, 2023, 11:44 PM
6 points
2
in reply to: gwd’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
I’ll look for the one that asked about the threat to humanity, and broke down responses by race and gender. In the meantime, here’s a poll showing general unease and bipartisan willingness to legally restrict the use of AI: https://web.archive.org/web/20180109060531/http://www.pewinternet.org/2017/10/04/automation-in-everyday-life/
Plus:
A SurveyMonkey poll on AI conducted for USA TODAY also had overtones of concern, with 73% of respondents saying that would prefer if AI was limited in the rollout of newer tech so that it doesn’t become a threat to humans.
Meanwhile, 43% said smarter-than-humans AI would do “more harm than good,” while 38% said it would result in “equal amounts of harm and good.”
I do note, on the other side, that the general public seems more willing to go Penrose, sometimes expressing or implying a belief in quantum consciousness unprompted. That part is just my own impression.

hairyfigment Mar 30, 2023, 9:06 PM
14 points
6
in reply to: gwd’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
The average person on the street is even further away from this I think.
This contradicts the existing polls, which appear to say that everyone outside of your subculture is much more concerned about AGI killing everyone. It looks like if it came to a vote, delaying AGI in some vague way would win by a landslide, and even Eliezer’s proposal might win easily.

hairyfigment Mar 30, 2023, 9:00 PM
2 points
0
in reply to: evhub’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
It would’ve been even better for this to happen long before the year of the prediction mentioned in this old blog-post, but this is better than nothing.

hairyfigment Mar 30, 2023, 8:07 PM
1 point
4
in reply to: Thoth Hermes’s comment on: Pausing AI Developments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky
Because the United Nations is a body chiefly concerned with enforcing international treaties, I imagine it would be incentivized to support arguments in favor of increasing its own scope and powers.
You imagine falsely, because your premise is false. The UN not only isn’t a body, its actions are largely controlled by a “Security Council” of powerful nations which try to serve their own interests (modulo hypotheticals about one of their governments being captured by a mad dog) and have no desire to serve the interests of the UN as such. This is mostly by design. We created the UN to prevent world wars, hence it can’t act on its own to start a world war.

hairyfigment Mar 30, 2023, 7:51 PM
2 points
0
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
I don’t know that I follow. The question, here and in the context of Löb’s Theorem, is about hypothetical proofs. Do you trust yourself enough to say that, in the hypothetical where you experience a proof that eating babies is mandatory, that would mean eating babies is mandatory?

hairyfigment Mar 30, 2023, 7:47 PM
2 points
0
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
I don’t even understand why you’re writing □(□P→P)→□P→P, unless you’re describing what the theory can prove. The last step, □P→P, isn’t valid in general, that’s the point of the theorem! If you’re describing what formally follows from full self-trust, from the assumption that □P→P for every P, then yes, the theory proves lots of false claims, one of which is a denial that it can prove 2+2=5.

hairyfigment Mar 30, 2023, 5:39 AM
2 points
0
in reply to: Thoth Hermes’s comment on: Why do the Sequences say that “Löb’s Theorem shows that a mathematical system cannot assert its own soundness without becoming inconsistent.”?
If you’re asking how we get to a formal self-contradiction, replace “False” with a statement the theory already disproves, like “0 does not equal 0.” I don’t understand your conclusion above; the contradiction comes because the theory already proves ¬False and now proves False, so the claim about “¬□False” seems like a distraction.