Mart_Korz

Karma: 256

Mart_Korz Feb 20, 2025, 4:50 PM
4 points
−5
on: Go Grok Yourself
Grok 3 told me 9.11 > 9.9. (common with other LLMs too), but again, turning on Thinking solves it.

This is unrelated to Grok 3, but I am not convinced that the above part of Andrej Karpathy’s tweet is a “gotcha”. Software version numbers use dots with a different meaning than decimal numbers and there 9.11 > 9.9 would be correct.
I don’t think there is a clear correct choice of which of these contexts to assume for an LLM if it only gets these few tokens.

E.g. if I ask Claude, the pure “is 9.11>9.9″ question gives me a no, whereas
”I am trying to install a python package. Could you tell me whether `9.11>9.9`?” gives me a yes.

Mart_Korz Feb 7, 2025, 4:48 PM
1 point
0
on: C’mon guys, Deliberate Practice is Real
For me, a strong reason why I do not see myself^[1] doing deliberate practice as you (very understandably) suggest is that, on some level, the part of my mind which decides on how much motivational oomph and thus effort is put into activities just in fact does not care much about all of these abstract and long-term goals.
Deliberate practice is a lot of hard work and the part of my mind which makes decisions about such levels of mental effort just does not see the benefits. There is a way in which a system that circumvents this motivational barrier is working against my short-term goals and and it is the latter who significantly controls motivation: Thus, such a system will “just sort of sputter and fail” in such a way that, consciously, I don’t even want to think about what went wrong.
If Feedbackloop Rationality wants to move me to be more rational, it has to work with my current state of irrationality. And this includes my short-sighted motivations.
And I think you do describe a bunch of the correct solutions: Building trust between one’s short-term motivations and long-term goals. Starting with lower-effort small-scale goals where both perspectives can get a feel for what cooperation actually looks like and can learn that it can be worth the compromises. In some sense, it seems to me that once one is capable of the kind of deliberate practice that you suggest, much of this boot strapping of agentic consistency between short-term motivation and deliberate goals has already happened.
On the other hand, it might be perfectly fine if Feedbackloop Rationality requires some not-yet-teachable minimal proficiency at this which only a fraction of people already have. If Feedbackloop Rationality allows these people to improve their thinking and contribute to hard x-risk problems, that is great by itself.
1. ^
  To some degree, I am describing an imaginary person here. But the pattern I describe definitely exists in my thinking even if less clearly than I put it above.

Mart_Korz Jan 29, 2025, 7:17 PM
4 points
0
on: Fake thinking and real thinking
Thank you for another beautiful essay at real thinking! This time about the mental stance itself.
But I’ll describe a few tags I’m currently using, when I remind myself to “really think.” Suggestions/tips from readers would also be welcome.
I think there is a strong conceptual overlap with what John Vervaeke describes as Relevance Realisation and wisdom.
I’ll attempt a summary of my understanding of John Vervaeke’s Relevance Realisation.
A key capability of any agentic/living beings is to prune the exponentially exploding space of possibilities when making any decision or thought. We are computationally bounded, and how we deal with that is crucial. Vervaeke terms the process that does this Relevance Realisation.
There is a lot of detail to his model, but let’s jump to how some of this plays out in human thinking: A core aspect in how we are agentic is our use of memeplexes that form an “agent-arena-relationship”—we combine a worldview with an agent that is suited to that world and then tie our identity to that agent. We build toy versions all the time (many games are like this), but—according to Vervaeke’s theses in the “Awakening from the Meaning Crisis” lectures—modern western culture has somewhat lost track of the cultural structures which allow individuals to coordinate on and (healthily) grow their mind’s agent-arena-relationship. We have institutions of truth (how to reach good factual statements rather than falsehoods), but not of wisdom (how to live with a healthy stance towards reality rather than bullshit. Historically religious institutions played that role but they now successfully do that for fewer and fewer people)
Inhabiting a functional such relationship feels alive and vibrant (“inhabit” = tying one’s identity into these memes), whereas the lack of a functional agent-arena-relationship feels dream-like (zombies are a good representation; maybe “NPC” is a more recent meme that points at this).
A related thing is people having a spiritual experience: this involves glimpsing a new agent-arena-relationship which then sometimes gets nourished into a re-structuring of one’s self-concept and priorities.
tying this back to “real thinking”
Although the process I described is not the same thing as real thinking, I do think that there are important similarities.
Regarding how to do this well, one important point of Vervaeke’s is that humans necessarily enter the world with very limited concepts of “self”, “agent” or “arena”. This perspective makes it clear that a core part of what we do while growing up is refining these concepts. A whole lot of our nature is about doing this process of transcending our previous self-concept. Verveake loves the quote “the sage is to the adult as the adult is to the child” to point at what the word wisdom means.
The process according to his recommendations, involves
- a Community of Practice (a community of people who share the goal of re-orienting themselves. One such is forming with and around him at awakentomeaning.com),
- an Ecology of Practices (meditation, reflecting on philosophy, embodiment practices, etc. are all contributing factors to reducing self-delusion; the term practice emphasizes that it is about a regular activity rather than about fixed beliefs). A good way to get a feel for his perspective might be the first episode(s) of his After Socrates youtube series
and from my impression, a lot of this is practicing the ability of inhabiting/changing the currently active agent-arena-perspective, exploring its boundaries, not flinching away from noticing its limitations, engaging with the perspectives which others have built and so on. Generally, a kind of fluidity in the mental motions which are involved in this.
I hope my descriptions are sufficiently right to give an impression of his perspective and whether there are some ideas that are valuable to you :)

Mart_Korz Oct 20, 2024, 2:19 PM
4 points
1
on: Arithmetic is an underrated world-modeling technology
It happens to be the case that 1 kWh = 3,600,000 kg/s². You could substitute this and cancel units to get the same answer.
This should be kg m²/s².^[1] On the other hand, this is a nice demonstration of how useful computers are: it really is too easy to accidentally drop a term when converting units
1. ^
  I double checked

Mart_Korz May 27, 2024, 8:49 PM
2 points
0
in reply to: reconstellate’s comment on: Truthseeking is the ground in which other principles grow
I read

to have a dude clearly describe a phenomenon he is clearly experiencing as “mostly female”

as “he makes a claim about a thought pattern he considers mostly female”, not as “he himself is described by the pattern” (QC does demonstrate high epistemic confidence in that post). Thus, I don’t think that Elizabeth would disagree with you.

Mart_Korz May 26, 2024, 6:15 PM
3 points
0
in reply to: johnswentworth’s comment on: How We Picture Bayesian Agents
Thanks for the guidance! Together with Gwern’s reply my understanding now is that caching can indeed be very fluidly integrated into the architecture (and that there is a whole fascinating field that I could try to learn about).

After letting the ideas settle for a bit, I think that one aspect that might have lead me to think

In my mind, there is an amount of internal confusion which feels much stronger than what I would expect for an agent as in the OP

is that a Bayesian agent as described still is (or at least could be) very “monolithic” in its world model. I struggle with putting this into words, but my thinking feels a lot more disjointed/local/modular. It would make sense if there is a spectrum from “basically global/serial computation” to “fully distributed/parallel computation” where going more to the right adds sources of internal confusion.

Mart_Korz May 26, 2024, 5:38 PM
1 point
0
in reply to: gwern’s comment on: How We Picture Bayesian Agents
What a reply, thank you!

Mart_Korz Apr 13, 2024, 1:12 PM
1 point
0
on: Ulm, Germany—ACX Spring Meetups Everywhere 2024
We are sitting close to the playground on top of red and blue blankets

Mart_Korz Apr 13, 2024, 12:29 PM
3 points
−2
in reply to: gilch’s comment on: Any evidence or reason to expect a multiverse / Everett branches?
Hmm.. . In my mind, the Pilot wave theory position does introduce a substrate dependence for the particle-position vs. wavefunction distinction, but need not distinguish any further than that. This still leaves simulation, AI-consciousness and mind-uploads completely open. It seems to me that the Pilot wave vs. Many worlds question is independent of/orthogonal to these questions.

I fully agree that saying “only corpuscle folk is real” (nice term by the way!) is a move that needs explaining. One advantage of Pilot wave theory is that one need not wonder about where the Born probabilities are coming from—they are directly implied of one wishes to make predictions about the future. One not-so-satisfying property is that the particle positions are fully guided by the wavefunction without any influence going the other way. I do agree that this makes it a lot easier to regard the positions as a superfluous addition that Occam’s razor should cut away.

For me, an important aspect of these discussions is that we know that our understanding is incomplete for every of these perspectives. Gravity has not been satisfyingly incorporated into any of these. Further, the Church-Turing thesis is an open question.

Mart_Korz Apr 12, 2024, 10:04 PM
1 point
−2
in reply to: evhub’s comment on: Any evidence or reason to expect a multiverse / Everett branches?
I am not too familiar with how advocates of Pilot wave theory usually state this, but I want to disagree slightly. I fully agree with the description of what happens mathematically in Pilot wave theory, but I think that there is a way in which the worlds that one finds oneself outside of do not exist.

If we assume that it is in fact just the particle positions which are “reality”, the only way in which the wave function (including all many worlds contributions) affects “reality” is by influencing its future dynamics. Sure, this means that the many worlds computationally do exist even in pilot wave theory. But I find the idea that “the way that the world state evolves is influenced by huge amounts of world states that ‘could have been’” meaningfully different to “there literally are other worlds that include versions of myself which are just as real as I am”. The first is a lot closer to everyday intuitions.

Well, this works to the degree to which we can (arbitrarily?) decide the particle positions to define “reality” (the thing in the theory that we want to look at in order to locate ourselves in the theory) in a way that is separate from being computationally a part of the model. One can easily have different opinions on how plausible this step is.

Mart_Korz Apr 9, 2024, 6:59 PM
LW: 4 AF: 3
0
AF
on: How We Picture Bayesian Agents

Finally, if we want to make the model capture certain non-Bayesian human behaviors while still keeping most of the picture, we can assume that instrumental values and/or epistemic updates are cached. This creates the possibility of cache inconsistency/incoherence.

In my mind, there is an amount of internal confusion which feels much stronger than what I would expect for an agent as in the OP. Or is the idea possibly that everything in the architecture uses caching and instrumental values? From reading, I imagined a memory+cache structure instead of being closer to “cache all the way down”.

Apart from this, I would bet that something interesting will happen for a somewhat human-comparable agent with regards to self-modelling and identity. Would anything similar to human identity emerge or would this require additional structure? Some representation of the agent itself, and its capabilities should be present at least

Mart_Korz Jul 7, 2023, 7:43 PM
4 points
0
on: Introducing bayescalc.io
After playing around für a few minutes, I like your app with >95% Probability ;) compare this bayescalc.io calculation

Mart_Korz Jun 28, 2023, 4:18 PM
9 points
0
in reply to: Jon Garcia’s comment on: Another medical miracle
Unfortunately, I do not have useful links for this—my understanding comes from non-English podcasts of a nutritionist. Please do not rely on my memory, but maybe this can be helpful for localizing good hypotheses.

According to how I remember this, one complication of veg*n diets and amino acids is that the question of which of the amino acids can be produced by your body and which are essential can effectively depend on your personal genes. In the podcast they mentioned that especially for males there is a fraction of the population who totally would need to supplement some “non-essential” amino acids if they want to stay healthy and follow veg*n diets. As these nutrients are usually not considered as worthy of consideration (because most people really do not need to think about them separately and also do not restrict their diet to avoid animal sources), they are not included in usual supplements and nutrition advice
(I think the term is “meat-based bioactive compounds”).

I think Elizabeth also emphasized this aspect in this post

Mart_Korz Jun 28, 2023, 3:49 PM
1 point
2
in reply to: niplav’s comment on: Self-Blinded Caffeine RCT

log score of my pill predictions (-0.6)

If did not make a mistake, this score could be achieved by e.g. giving ~55% probabilities and being correct every time or by always giving 70% probabilities and being right ~69 % of the time.

Mart_Korz Jun 28, 2023, 3:28 PM
4 points
0
in reply to: niplav’s comment on: Self-Blinded Caffeine RCT

you’d expect the difference in placebo-caffeine scores to drop

I am not sure about this. I could also imagine that the difference remains similar, but instead the baseline for concentration etc. shifts downwards such that caffeine-days are only as good as the old baseline and placebo-days are worse than the old baseline.

Mart_Korz Jun 25, 2023, 7:41 PM
4 points
0
in reply to: Mart_Korz’s comment on: Steering GPT-2-XL by adding an activation vector
Update: I found a proof of the “exponential number of near-orthogonal vectors” in these lecture notes https://www.cs.princeton.edu/courses/archive/fall16/cos521/Lectures/lec9.pdf From my understanding, the proof uses a quantification of just how likely near-orthogonality becomes in high-dimensional spaces and derives a probability for pairwise near-orthogonality of many states.

This does not quite help my intuitions, but I’ll just assume that the “it it possible to tile the surface efficiently with circles even if their size gets close to the 45° threshold” resolves to “yes, if the dimensionality is high enough”.

One interesting aspect of these considerations should be that with growing dimensionality the definition of near-orthogonality can be made tighter without loosing the exponential number of vectors. This should define a natural signal-to-noise ratio for information encoded in this fashion.

Mart_Korz Jun 23, 2023, 8:22 PM
2 points
0
in reply to: Carl Feynman’s comment on: Steering GPT-2-XL by adding an activation vector
Weirdly, in spaces of high dimension, almost all vectors are almost at right angles.
This part, I can imagine. With a fixed reference vector written as $(1, 0, 0, \dots, 0)$ , a second random vector has many dimensions that it can distribute its length along $(x_{1}, x_{2}, x_{2}, \dots, x_{d})$ while for alignment to the reference (the scalar product) only the first entry $x_{1}$ contributes.
It’s perfectly feasible for this space to represent zillions of concepts almost at right angles to each other.
This part I struggle with. Is there an intuitive argument for why this is possible?
If I assume smaller angles below 60° or so, a non-rigorous argument could be:
- each vector blocks a 30°-circle around it on the d-hypersphere^[1] (if the circles of two vectors touch, their relative angle is 60°).
- an estimate for the blocked area could be that this is mostly a ‘flat’ (d-1)-sphere of radius $30 ° / (1 rad) = π / 6 \approx 0.6$ which has an area that scales with $A_{v e c t o r} \sim (0.6)^{d - 1}$
- the full hypersphere has a surface area with a similar pre-factor but full radius $A \sim (1)^{d - 1}$
- thus we can expect to fit a number of vectors $N$ that scales roughly like $N \sim A / A_{v e c t o r} \sim (\frac{1}{0.6})^{d - 1}$ which is an exponential growth in $d$ .
For a proof, one would need to include whether it is possible to tile the surface efficiently with the $A_{v e c t o r}$ circles. This seems clearly true for tiny angles (we can stack spheres in approximately flat space just fine), but seems a lot less obvious for larger angles. For example, full orthogonality would mean 90° angles and my estimate would still give $N \sim (\frac{1}{π / 4})^{d - 1} \approx (1.27)^{d - 1}$ , an exponential estimate for the number of strictly orthogonal states although these are definitely not exponentially many.
and a copy of that circle on the opposite end of the sphere ↩︎

Mart_Korz Jun 19, 2023, 8:56 PM
1 point
on: Why Doesn’t Healthcare Improve Health?
But the best outcomes seem to come out of homeopathy, which is as perfect of a placebo arm as one can get.

I did expect to be surprised by the post given the title, but I did not expect this surprise.

I have previously heard lots of advocates for evidence-based medicine claim that homoeopathy has very weak evidence for effects (mostly the amount that one would expect from noise and flawed studies, given the amount of effort being put into proving its efficacy) – do I understand correctly that this is an acceptable interpretation, while the aggregate mortality of real-world patients (as opposed to RCT participants) clearly improves when treated homoeopathically (compared to usual medicine)?

More generally, if I assume that the shift “no free healthcare”->”free healthcare” does not improve outcomes, and that “healthcare”->”healthcare+homoeopathy” does improve outcomes, wouldn’t that imply that “healthcare+homoeopathy” is preferable to “no free healthcare”?
- of course, there are a lot of steps in this argument that can go wrong
- but generally, I would expect that something like this reasoning should be right.
- if I do assume that homoeopathy is practically a placebo, this can point us to at least some fraction of treatments which should be avoided: those that homoeopathy claims to heal without the need for other treatments
That looks like an argument that an approach like your “What I do”-section can actually lead to strong benefits from the health system, and that non-excessively complicated strategies are available.

Mart_Korz Jun 16, 2023, 11:03 AM
4 points
0
in reply to: avturchin’s comment on: I still think it’s very unlikely we’re observing alien aircraft
One aspect which I disagree with is that collapse is the important thing to look at. Decoherence is sufficient to get classical behaviour on the branches of the wave function. There is no need to consider collapse if we care about ‘weird’ vs. classical behaviour. This is still the case even if the whole universe is collapse-resistant (as is the case in the many worlds interpretation). The point of this is that true cat states ( = superposed universe branches) do not look weird.

The whole ‘macroscopic quantum effects’ are interferences between whole universes branches from the view of this small quantum object in they brain.

Superposition of universe—We can certainly regard the possibility that the macroscopic world is in a superposition as seen from our brain. This is what we should expect (absent collapse) just from the sizes of universe and brain:
1. The size of our brain corresponds to a limited number for the dimensionality of all possible brain states (we can include all sub-atomic particles for this)
2. If the number of branches of the universe is larger than the number of possible brain states, there is no possible wave function in which there aren’t some contributions in which the universe is in a superposition with regards to the brain. Some brain states must be associated with multiple branches.
3. the universe is a lot larger than the brain and dimensionality scales exponentially with particle number
4. further, it seems highly likely that many physical brain-states correspond to identical mind states (some unnoticeable vibration propagating through my body does not seem to scramble my thinking very much)
Because of this, anyone following the many worlds interpretation should agree that from our perspective, the universe is always in a superposition—no unknown brain properties required. But due to decoherence (and assuming that branches will not meet), this makes no difference and we can replace the superposition with a probability distribution.

Perhaps this is captured by your “why Everett called his theory relative interpretation of QM”—I did not read his original works.

The question now becomes the interference between whole universe branches: A deep assumption in quantum theory is locality which implies that two branches must be equal in all properties^[1] in order to interfere^[2]. Because of this, interference of branches can only look like “things evolving in a weird direction” (double slit experiment) and not like “we encounter a wholly different branch of reality” (fictional stories where people meet their alternate-reality versions).

Because of this, I do not see how quantum mechanics could create the weird effects that it is supposed to explain.

If we do assume that human minds have an extra ability to facilitate interaction between otherwise distant branches if they are in a superposition compared to us, this of course could create a lot of weirdness. But this seems like a huge claim to me that would depart massively from much of what current physics believes. Without a much more specific model, this feels closer to a non-explanation than to an explanation.
1. ↩︎
  more strictly: must have mutual support in phase-space. For non-physicists: a point in phase-space is how classical mechanics describes a world.
2. ↩︎
  This is not a necessary property of quantum theories, but it is one of the core assumptions used in e.g. the standard model. People who explore quantum gravity do consider theories which soften this assumption

Mart_Korz Jun 16, 2023, 9:04 AM
1 point
0
in reply to: dr_s’s comment on: Ethodynamics of Omelas
I mean, if it’s about looking for post-hoc rationalizations, what’s even the point of pretending there’s a consistent ethical system?

Hmm, I would not describe it as rationalization in the motivated reasoning sense.

My model of this process is that most of my ethical intuitions are mostly a black-box and often contradictory, but still in the end contain a lot more information about what I deem good than any of the explicit reasoning I am capable of. If however, I find an explicit model which manages to explain my intuitions sufficiently well, I am willing to update or override my intuitions. I would in the end accept an argument that goes against some of my intuitions if it is strong enough. But I will also strive to find a theory which manages to combine all the intuitions into a functioning whole.

In this case, I have an intuition towards negative utilitarianism, which really dislikes utility monsters, but I also have noticed the tendency that I land closer to symmetric utilitarianism when I use explicit reasoning. Due to this, the likely options are that after further reflection I
- would be convinced that utility monsters are fine, actually.
- would come to believe that there are strong utilitarian arguments to have a policy against utility monsters such that in practice they would almost always be bad
- would shift in some other direction
and my intuition for negative utilitarianism would prefer cases 2 or 3.

So the above description was what was going on in my mind, and combined with the always-present possibility that I am bullshitting myself, led to the formulation I used :)