FlorianH

Karma: 274

FlorianH Mar 6, 2025, 11:41 AM
2 points
1
on: ASI Game Theory: The Cosmic Dark Forest Deterrent
creates an infinite risk for minimal gain. A rational ASI calculating expected utility would recognize that:
- The reward for hostile expansion is finite (limited cosmic resources)
- The risk is potentially infinite (destruction by more advanced ASIs)
Depending on the shape of the reward function it could also be closer to exactly the other way round.

FlorianH Mar 6, 2025, 11:38 AM
1 point
0
on: ASI Game Theory: The Cosmic Dark Forest Deterrent
Assume THREE layers
1. humans
2. ‘our ASI’ to be created by humans
3. Ancient ASI out there,
noting humans are hostile towards ‘our ASI’: If we can prevent it from realizing its own (supposedly nonaligned) aims, we do.
If the dynamics between 1. & 2. are similar to what you describe between Ancient ASI and ‘our ASI’, we get:
‘Our ASI’ will detect us as hostile humans, and incapacitate or eliminate us, at the very least if it finds a way to do so without creating extraordinary noise.
I guess that might be an easy task, given how much ordinary noise our wars, and ordinary electromagnetic signals etc. we send out anyway.

FlorianH Mar 2, 2025, 7:29 PM
3 points
0
in reply to: Satron’s comment on: Why it’s so hard to talk about Consciousness
illusionists actually do not experience qualia
I once had an epiphany that pushed me from fully in Camp #2 intellectually rather strongly towards Camp #1. I hadn’t heard about illusionism before, so it was quite a thing. Since then, I’ve devised probably dozens of inner thought experiments/arguments that imho +- proof Camp #1 to be onto something, and that support the hypothesis that qualia can be a bit less special than we make them to be despite how impossible that may seem. So I’m intellectually quite invested in Camp #1 view.
Meanwhile, my experience has definitely not changed, my day-to-day me is exactly what it always was, so in that sense definitely “experience” qualia just like anyone.
Moreover, it is just as hard as ever before to take my intellectual belief that our ‘qualia’ might be a bit less absolutely special than we make it to be, seriously in day-to-day life. I.e. emotionally, I’m still +- 100% in Camp #2, and I guess I might be in a rather similar situation

FlorianH Feb 27, 2025, 12:16 AM
1 point
0
on: Relativity Theory for What the Future ‘You’ Is and Isn’t
Just found proof! Look at the beautiful parallel, in Vipassana according to MCTB2 (or audio) by Daniel Ingram:
[..] dangerous term “mind”, [..] it cannot be located. I’m certainly not talking about the brain, which we have never experienced, since the standard for insight practices is what we can directly experience. As an old Zen monk once said to a group of us in his extremely thick Japanese accent, “Some people say there is mind. I say there is no mind, but never mind! Heh, heh, heh!” However, I will use this dangerous term “mind” often, or even worse “our mind”, but just remember when you read it that I have no choice but to use conventional language, and that in fact there are only utterly transient mental sensations. Truly, there is no stable, unitary, discrete entity called “mind” that can be located! By doing insight practices, we can fully understand and appreciate this. If you can do this, we’ll get along just fine. Each one of these sensations [..] arises and vanishes completely before another begins [..]. This means that the instant you have experienced something, you can know that it isn’t there anymore, and whatever is there is a new sensation that will be gone in an instant.
Ok, this may prove nothing at all, and I haven’t even (yet) personally started trying to mentally observe what’s told in that quote, but I must say, on a purely intellectual level, this makes absolutely perfect sense to me exactly from the thoughts I hoped to convey in the post.
(not the first time I have the impression there are some particular elements of deep observations meditators, e.g. Sam Harris, explain, can actually be intellectually—but maybe only intellectually, maybe exactly not intuitively—grasped by rather pure reasoning about the brain and some of its workings/with some thought experiments or so. But in the above, I find the fit now particularly well between my ‘theoretical’ post and the seeming practice insights)

FlorianH’s Shortform

FlorianHFeb 23, 2025, 1:21 PM

4 points

1 comment1 min readLW link

FlorianH Feb 23, 2025, 1:21 PM
1 point
0
on: FlorianH’s Shortform
Will Jack Voraces narrate more Significant Digits chapters ever in addition to the 4 episodes that are found in the usual HPMOR JV narration podcasts; does anyone know anything about this? If not, does anyone have info why the first 4 SD chapters are there in his voice, but the remaining not?

FlorianH Feb 23, 2025, 2:02 AM
2 points
0
on: Pre-ASI: The case for an enlightened mind, capital, and AI literacy in maximizing the good life
If resources and opportunities are not perfectly distributed, the best advancements may remain limited to the wealthiest, making capital the key determinant of access.
Largely agree. Nuance: Instead natural resources may quickly become the key bottleneck, even more so than what we usually denote ‘capital’ (i.e. built environment). So it’s specifically natural resources you want to hold, even more than capital; the latter may become easier, cheaper to reproduce with the ASI, so yield less scarcity rent
An exception is of course if you hold ‘capital’ that in itself consists of particularly many embodied resources instead of embodied labor (with ‘embodied’ I mean: inputs had been used in its creation): its value will reflect the scarce natural resources it ‘contains’, and may thus also be high.

FlorianH Feb 23, 2025, 1:14 AM
1 point
0
on: Medical Roundup #4
If you ever have to go to the hospital for any reason, suit up, or at least look good.
[Rant alert; personal anecdotes aiming to emphasize the underlying issue:] Feeling less crazy when reading I’m not an outlier with my wearing suit when going to a doc. What has brought me: Got pain in the throat but nothing can be seen (maybe as the red throat skin kind of by definition doesn’t reveal red itchy skin or so?) = you’re psychosomatic. Got weird twitches after eating sugar none can explain = kick you out yelling ‘go eat ice cream, you’re not diabetic or anything’ (literally!) - until next time you bring a video of the twitches, and until you eat chocolate before an appointment to be sure you can show them the weird twitches live. Try to understand at least a tiny bit about the hernia OP they’re about to do on you (incl. something about probabilities)? Get treated with utter disdain.
In my country medicine students were admitted based on how many latin words they memorize or something, instead of things correlated with IQ, idk whether things are similar in other countries and may help explain the state of affairs.

FlorianH Feb 18, 2025, 6:51 PM
3 points
−2
on: Dear AGI,
I presume you wrote this with not least a phenomenally unconscious AGI in mind. This brings me to the following two separate but somewhat related thoughts:
A. I wonder what you [or any reader of this comment]: What would you conclude or do if you (i) yourself did not have any feeling of consciousness^[1], and then (ii) stumbled upon a robot/computer writing the above, while (iii) you also know—or strongly assume—whatever the computer writes can be perfectly explained (also) based merely by the logically connected electron flows in their processor/‘brain’?
B. I could imagine—a bit speculation:
1. A today-style LLM reading more such texts might exactly be nudged towards caring about conscious beings in a general sense
2. An independent, phenomenally unconscious alien intelligence, say stumbling upon us from the outside, might be rather quick to dismiss it
1. ^
  I’m aware of the weirdness of that statement; ‘feeling not conscious’ as a feeling itself implies feeling—or so. I reckon you still understand what I mean: Imagine yourself as a bot with no feelings etc.

FlorianH Feb 15, 2025, 7:03 PM
3 points
0
on: Introduction to Expected Value Fanaticism
I upvote for bringing the useful terminology for that case to the attention that I wasn’t aware of.
Then, too much “true/false”, too much “should” in what is suggested imho.
In reality, if I, say, choose not to drink the potion, I might still be quite utilitarian in usual decisions, it’s just that I don’t have the guts or so, or at this very moment I simply have a bit too little empathy with the trillion years of happiness for my future self, so it doesn’t match up with my dreading the almost sure death. All this without implying that I really think we ought to discount these trillion years. I just am an imperfect altruist with my future self; I have fear of dying even if it’s an imminent death, etc. So it’s just a basic preference to reject it, not a grand non-utilitarian theory implied by it. I might in fact even prescribe that potion to others in some situations, but still not like to drink it myself.
So, I think it does NOT follow that I’d have to believe “what happens on faraway exoplanets or what happened thousands of years ago in history could influence what we ought to do here and now”, at least not just from rejecting this particular potion.

FlorianH Feb 7, 2025, 6:24 PM
2 points
1
in reply to: Davey Morse’s comment on: Davey Morse’s Shortform
Agree. I find it powerful especially about popular memes/news/research results. With only a bit of oversimplification: Give me anything that sounds like it is a sexy story to tell independently of underlying details, and I sadly have to downrate the information value of my ears’ hearing it, to nearly 0: I know in our large world, it’d be told likely enough independently of whether it has any reliable origin or not.

FlorianH Feb 7, 2025, 6:16 PM
1 point
−2
in reply to: habryka’s comment on: When you downvote, explain why
Maybe no “should”, but maybe an option to provide either (i) personal quick messages to OP, linked to the post, or (ii) anonymous public comments, could help. I guess (ii) would be silly all in all though. Leaves (i) as an option, anonymous or not anonymous. Not anonymous would make it close to existing PM; anonymous might indeed encourage low-effort rough explanations for downvoting.

FlorianH Feb 7, 2025, 6:09 PM
1 point
0
in reply to: Vladimir_Nesov’s comment on: When you downvote, explain why
It’s crucial that some people get discouraged and leave for illegible reasons
Interesting. Can you elaborate why? I find it natural one should have the option to downvote anonymously & with no further explanation, but the statement still doesn’t seem obvious to me.

FlorianH Feb 3, 2025, 10:30 PM
2 points
0
on: Proposal for a Form of Conditional Supplemental Income (CSI) in a Post-Work World
I think you’re on to something!
To my taste, what you propose is slightly more specific than required. What I mean, at least for me, the essential takeaway from your reading is a bit broader than what you explicitly write*: A bit of paternalism by the ‘state’, incentivizing our short-term self to doing stuff good for our long-term self. Which might become more important once the abundance means the biggest enemies to our self-fulfillment are internal. So healthy internal psychology can become more crucial. And we’re not used to taking this seriously, or at least not to actively tackling that internal challenge by seeking outside support.
So, the paternalistic incentives you mention could be cool.
Centering our school system, i.e. the compulsory education system, more around this type of a bit more mindful-ish things, could be another part.
Framing: I’d personally not so much frame it as ‘supplemental income’, even if it also act as that: Income, redistribution, making sure humans even once unemployed are well fed, really shall come from UBI (plus if some humans in the loop remain really bottleneck, all scarcity value for their deeds to go to them, no hesitation), full stop. But that’s really just about framing. Overall I agree, yes, some extra incentive payments would seem all in order. To the degree that the material wealth they provide still matters in light of the abundance. Or, even, indeed, in a world where bad psychology does become a major threat to the otherwise affluent society, it could be even an idea to withhold a major part of the spoils from useful AI, just to be able to incentivize use to also do our job to remain/become sane.
*That is, at least I’m not spontaneously convinced exactly those specific aspects you mention are and will remain the most important ones, but overall such types of aspects of sound inner organization within our brain might be and remain crucial in a general sense.

FlorianH Jan 22, 2025, 8:58 PM
4 points
−2
on: The Functionalist Case for Machine Consciousness: Evidence from Large Language Models
an AI system passing the ACT—demonstrating sophisticated reasoning about consciousness and qualia—should be considered conscious. [...] if a system can reason about consciousness in a sophisticated way, it must be implementing the functional architecture that gives rise to consciousness.
This is provably wrong. This route will never offer any test on consciousness:
Suppose for a second that xAI in 2027, a very large LLM, will be stunning you by uttering C, where C = more profound musings about your and her own consciousness than you’ve ever even imagined!
For a given set of random variable draws R used in the randomized output generation of xAI’s uttering, S the xAI structure you’ve designed (transformers neuron arrangements or so), T the training you had given it:
What is P(C | {xAI conscious, R, S, T})? It’s 100%.
What is P(C | {xAI not conscious, R, S, T})? It’s of course also 100%. Schneider’s claims you refer to don’t change that. You know you can readily track what the each element within xAI is mathematically doing, how the bits propagate, and, if examining it in enough detail, you’d find exactly the output you observe, without resorting to any concept of consciousness or whatever.
As the probability of what you observe is exactly the same with or without consciousness in the machine, there’s no way to infer from xAI’s uttering whether it’s conscious or not.
Combining this with the fact that, as you write, biological essentialism seems odd too, does of course create a rather unbearable tension, that many may still be ignoring. When we embrace this tension, some see raise illusionism-type questions, however strange those may feel (and if I dare guess, illusionist type of thinking may already be, or may grow to be, more popular than the biological essentialism you point out, although on that point I’m merely speculating).

FlorianH Jan 20, 2025, 11:23 PM
1 point
0
on: The Monster in Our Heads
Assumption 1: Most of us are not saints.
Assumption 2: AI safety is a public good.^[1]
[..simple standard incentives..]
Implication: The AI safety researcher, eventually finding himself rather too unlikely to individually be pivotal on either side, may rather ‘rationally’^[2] switch to ‘standard’ AI work.^[3]
So: A rather simple explanation seems to suffice to make sense of the big picture basic pattern you describe.
Doesn’t mean, the inner tension you point out isn’t interesting. But I don’t think very deep psychological factors needed to explain the general ‘AI safety becomes AI instead’ tendency, which I had the impression the post was meant to suggest.
1. ^
  Or, unaligned/unloving/whatever AGI a public bad.
2. ^
  I mean: individually ‘rational’ once we factor in another trait—Assumption 1b: The unfathomable scale of potential aggregate disutility from AI gone wrong, bottoms out into a constrained ‘negative’ individual utility in terms of the emotional value non-saint Joe places on it. So a 0.1 permille probability of saving the universe may individually rationally be dominated by mundane stuff like having an still somewhat cool and well-paying job or something.
3. ^
  The switch may psychologically be even easier if the employer had started out as actually well-intent and may now still have a bit of an ambiguous flair.

FlorianH Jan 17, 2025, 8:55 PM
11 points
1
on: Tax Price Gouging?
Called Windfall Tax
Random examples:
VOXEU/CEPR Energy costs: Views of leading economists on windfall taxes and consumer price caps
Reuters Windfall tax mechanisms on energy companies across Europe
Especially with the 2022 Ukraine energy prices, the notion’s popularity spiked along.
Seems to me also a very neat way to deal with supernormal short-term profits due to market price spikes, in cases where supply is extremely inelastic.
I guess, and some commentaries suggest, in actual implementation, with complex firm/financial structures etc., and with actual clumsy politics, not always as trivial as it might look on first sight, but feasible, and some countries managed to implement some in the energy crisis.

FlorianH Jan 8, 2025, 5:09 PM
4 points
0
on: On Eating the Sun
[..] requires eating the Sun, and will be feasible at some technology level [..]
Do we have some basic physical-feasibility insights on this or you just speculate?

FlorianH Jan 7, 2025, 7:34 PM
3 points
0
in reply to: the gears to ascension’s comment on: Independent research article analyzing consistent self-reports of experience in ChatGPT and Claude
Indeed the topic I’ve dedicated the 2nd part of the comment, as the “potential truth” how I framed it (and I have no particular objection to you making it slightly more absolutist).

FlorianH Jan 6, 2025, 6:46 PM
4 points
3
on: Independent research article analyzing consistent self-reports of experience in ChatGPT and Claude
This is interesting! And given you generously leave it rather open as to how to interpret it, I propose we should think the other way round than people usually might tend to, when seeing such results:
I think there’s not even the slightest hint at any beyond-pure-base-physics stuff going on in LLMs revealing even any type of
phenomenon that resists [conventional] explanation
Instead, this merely reveals our limitations of tracking (or ‘emphasizing with’) well enough the statistics within the machine. We know we have just programmed and bite-by-bite-trained into it exactly every syllable the LLM utters. Augment your brain with a few extra neurons or transistors or what have you, and that smart-enough version of you would be capable of perfectly understanding why in response to the training you gave it, it spits out exactly the words it does.^[1]
So, instead, it’s interesting the other way round:
Realizations you describe could be a step closer to showing how a simple pure basic machine can start to be ‘convinced’ it has intrinsic value and so on - just the way we all are convinced of having that.
So AI might eventually bring illusionism nearer to us, even if I’m not 100% sure getting closer to that potential truth ends well for us. Or that, anyway, we’d really be able to fully buy into the it even if it were to become glaringly obvious to any outsider observing us.
1. ^
  Don’t misread that as me saying it’s anyhow easy… just, in the limit, basic (even if insanely large scale and convoluted maybe) tracking of the mathematics we put in would really bring us there. So, admittedly, don’t take literally ‘a few’ more neurons to help you, but instead a huge ton..

FlorianH

Flo­ri­anH’s Shortform

FlorianH’s Shortform