Anon User

Karma: 524

Anon User Mar 24, 2025, 5:18 AM
3 points
0
on: We need (a lot) more rogue agent honeypots
There’s probably more. There should be more—please link in comments, if you know some!
Wouldn’t “outing” potential honeypots be extremely counterproductive? So yeah, if you know some—please keep it to yourself!

Anon User Mar 13, 2025, 7:42 PM
2 points
0
on: Why Obedient AI May Be the Real Catastrophe
Oftentimes downvoting without taking time to commet and explain reasons is reasonable, and I tends to strongly disagree with people who think I owe an incompetent write an explanation when downvoting. However, just this one time I would ask—can some of the people downvoting this explain why?
It is true that our standard way of mathematically modeling things implies that any coherent set of preferences must behave like a value function. But any mathematical model of the world is new essarily incomplete. A computationally limited agent that cannot fully foresee all consequences of its choices cannon have a coherent set of preferences to begin with. Should we be trying to figure out how to model computational limitations in a way that acknowledges that some form of preserving future choice might be an optimal strategy? Including preserving some future choice on how to extend the computationally limited objective function onto uncertain future situations?

Anon User Mar 4, 2025, 6:21 AM
5 points
0
on: How much should I worry about the Atlanta Fed’s GDP estimates?
This looks to be primarily about imports—that is, primarily taking into account Trump’s new tariffs. I am guessing that Wall Street does not quite believe that Trump actually means it...

Anon User Feb 16, 2025, 8:15 AM
3 points
0
in reply to: MattJ’s comment on: Any Trump Supporters Want to Dialogue?
It would seem that my predictions of how Trump would approach this were pretty spot on… @MattJ I am curious what’s your current take on it?

Anon User Feb 16, 2025, 8:11 AM
1 point
0
in reply to: Noosphere89’s comment on: Introduction to Expected Value Fanaticism
Why would the value to me personally of existence of happy people be linear in the number of them? Does creating happy person #10000001 [almost] identical to the previous 10000000 as joyous as when the 1st of them was created? I think value is necessary limited. There are always diminishing returns from more of the same...

Anon User Feb 6, 2025, 5:13 AM
5 points
0
in reply to: Alfred Harwood’s comment on: Ruling Out Lookup Tables
> if you have a program computing a predicate P(x, y) that is only true when y = f(x), and then the program just tries all possible y—is that more like a function, or more like a lookup?
In order to test whether y=f(x), the program must have calculated f(x) and stored it somewhere. How did it calculate f(x)? Did it use a table or calculate it directly?
What I meant is that the program knows how to check the answer, but not how to compute/find one, other than by trying every answer and then checking it. (Think: you have a math equation, no idea how to solve for x, so you are just trying all possible x in a row).

Anon User Feb 6, 2025, 5:10 AM
2 points
0
on: Alignment Paradox and a Request for Harsh Criticism
Aligned with current (majority) human values, meaning any social or scientific human progress would be stifled by the AI and humanity would be doomed to stagnate.
Only true when current values are taked naively, because future progress is a part of current human values (otherwise we would not be all agreeing with you that preventing it would be a bad outcome). It is hard to coherently generalize and extrapolate the human values, so that future progress is included in that, but not necessarily impossible.

Anon User Feb 4, 2025, 3:43 PM
2 points
0
on: How AGI Defines Its Self
Your timelines do not add up. Individual selection works on smaller time scales than group selection, and once we get to a stage of individual selection acting in any non-trivial way on AGI agents capable of directly affecting the outcomes, we already lost—I think at this point it’s pretty much a given that humanity is doomed on a lot shorter time scale that that required for any kinds of group selection pressures to potentially save us...

Anon User Feb 4, 2025, 3:09 PM
9 points
0
on: Ruling Out Lookup Tables
This seems to be making a somewhat arbitrary distinction—specifically a program that computes f(x) in some sort of a direct way, and a program that computes it in some less direct way (you call it a “lookup table”, but you seem to actually allow combining that with arbitrary decompression/decoding algorithms). But realistically, this is a spectrum—e.g. if you have a program computing a predicate P(x, y) that is only true when y = f(x), and then the program just tries all possible y—is that more like a function, or more like a lookup? What about if you have first compute some simple function of the input (e.g. x mod N), then do a lookup?

Anon User Feb 4, 2025, 2:46 PM
1 point
0
in reply to: Yair Halberstadt’s comment on: Stopping unaligned LLMs is easy!
Yes, and I was attempting to illustrate why this is a bad assumption. Yes, LLMs subject to unrealistic limitations are potentially easier to align, but that does not help, unfortunately.

Anon User Feb 3, 2025, 7:06 PM
1 point
0
on: Stopping unaligned LLMs is easy!
You ask a superintendent LLM to design a drug to cure a particular disease. It outputs just a few tokens with the drug formula. How do you use a previous gen LLM to check whether the drug will have some nasty humanity-killing side-effects years down the road?
Edited to add: the point is that even with a few tokens, you might still have a huge inferential distance that nothing with less intelligence (including humanity) could bridge.

Anon User Feb 2, 2025, 7:23 PM
−4 points
0
in reply to: Dagon’s comment on: How likely is an attempted coup in the United States in the next four years?
Agreed on your second part. A part of Trump “superpower” is to introduce a lot of confusion around the bounds, and then convince at least his supporters that he is not really stepping over that where it should have been obvious that he does. So the category “should have been plainly illegal and would have been considered plainly illegal before, but now nobody knows anymore” is likely to be a lot better defined that “still plainly illegal”. Moreover, Trump is much more likely to attempt the former than the latter—not because he actually cares about not doing the latter, but because anything he actually does has a tendency to be reclassified from latter to former. Including after the fact—e.g. many of his past actions were moved from the latter category to former one by the Supreme Court presidential immunity decision...

Anon User Jan 31, 2025, 12:06 AM
1 point
0
in reply to: samuelshadrach’s comment on: Do you consider perfect surveillance inevitable?
Yes, potentially less that ASI, and security is definitely an issue, But people breaching the security would hoard their access—there will be periodic high-profile spills (e.g. celebrities engaged in sexual activities, or politicians engaged in something inappropriate would be obvious targets), but I’d expect most of the time people would have at least an illusion of privacy.

Anon User Jan 31, 2025, 12:01 AM
4 points
2
on: Can someone, anyone, make superintelligence a more concrete concept?
I found Eliezer Yudkowsky’s “blinking stars” story (That Alien Message — https://search.app/uYn3eZxMEi5FWZEw5) persuasive. That story also has a second layer of having the extra smart Earth with better functioning institutions, but at the level of intuition you are going for it is probably unnecessary and would detract from the message. I think imagining a NASA-like organisation dedicated to controlling a remote robot at say 1 cycle of control loop per month (where it is perhaps corresponding to 1/30 of a second for the aliens), showing how totally screwed up the aliens are in this scenario, then flipping it around, should be at least somewhat emotionally persuasive.

Anon User Jan 24, 2025, 6:43 AM
1 point
0
on: A hierarchy of disagreement
For the specific example of arguing in a podcast, would not you expect people to already be aware of a substantial subset of arguments from the other side, and so would not it be entirely expected that there would be 0 update on information that is not new, and so not as much update overall, if only a fraction of information is actually new?

Anon User Jan 24, 2025, 6:37 AM
4 points
1
on: Do you consider perfect surveillance inevitable?
Hm, not sure about it being broadcast vs consumed by a powerful AI that somebody else has at least a partial control over.

Anon User Jan 19, 2025, 9:53 PM
1 point
0
in reply to: Mikhail Samin’s comment on: No one has the ball on 1500 Russian olympiad winners who’ve received HPMOR
Getting to the national math Olympiad requires access to regional Olympiad first, then being able to travel. Smart kids from “middle of nowhere” places—exactly to the kinds of kids you want to reach—are more likely to participate in the cities tournament. I wonder whether kids who were eligible for the summer camp, but did not make it there are more of your target audience than those who participated in the camp.
P.S. my knowledge of this is primarily based on how things were ~35 years ago, so I could be completely off.

Anon User Jan 12, 2025, 4:05 PM
6 points
1
on: No one has the ball on 1500 Russian olympiad winners who’ve received HPMOR
What about trying to use the existing infrastructure in Russia, e.g.
- Donating to school libraries of math magnet schools (starting with “usual suspects” of 57, 2, 43 in Moscow, 239 in St Petersburg, etc, and then going down the list)?
- Contacting a competition organizers (e.g. for тургор - турнир городов which tends to have a higher diversity of participants compared to the Olympiad system) and coordinating to use the books as prises for finalists?
Besides not having to reinvent the wheel, kids might be more open to the ideas if the book comes from a local, more readily trusted party.

Anon User 7 Jan 2025 3:40 UTC
3 points
1
in reply to: AynonymousPrsn123’s comment on: Is my distinctiveness evidence for being in a simulation?
Think MMORPGs—what are the chances of simulation being like that vs a simulation with just a few special beings, and the rest NPCs?. Even if you say it’s ⁵⁰⁄₅₀, then given that MMORPG-style simulations have billions of observes and “observers are special” ones only have a few, then an overwhelming majority of simulates observers are actually not that special in their simulations.

Anon User 25 Nov 2024 18:07 UTC
1 point
0
on: AI Specialized in ML Training Could Create ASI: AGI Is Unnecessary
https://www.lesswrong.com/tag/recursive-self-improvement