Mart_Korz

Karma: 256

Mart_Korz Feb 5, 2023, 7:05 PM
1 point
0
in reply to: pjeby’s comment on: What fact that you know is true but most people aren’t ready to accept it?
This time, I agree fully :)

Mart_Korz Feb 5, 2023, 9:09 AM
2 points
1
in reply to: pjeby’s comment on: What fact that you know is true but most people aren’t ready to accept it?
I think there is a lot of truth to this, but I do not quite agree.
Most long-lasting negative emotions and moods exist solely for social signaling purposes
feels a bit off to me. I think I would agree with an alternate version “most long-lasting negative emotions and moods are caused by our social cognition” (I am not perfectly happy with this formulation).
In my mind the difference is that “for signalling purposes” contains an aspect of a voluntary decision (and thus blame-worthiness for the consequences), whereas my model of this dynamic is closer to “humans are kind of hard-wired to seek high-calorie food which can lead to health problems if food is in abundance”. I guess many rationalists are already sufficiently aware that much of human decision-making (necessarily) is barely conscious. But I think that especially when dealing with this topic of social cognition and self-image it is important to emphasize that some very painful failure modes are bundled with being human and that, while we should take agency in avoiding/overcoming them, we do not have the ability to choose our starting point.

On a different note:
This Ezra Klein Show interview with Rachel Aviv has impressive examples of how influential culture/memes can be for mental (and even physical) illnesses and also how difficult it is to culturally deal with this.

Mart_Korz Feb 1, 2023, 10:54 PM
1 point
0
on: Schizophrenia as a deficiency in long-range cortex-to-cortex communication
I think my hypothesis would more naturally predict that schizophrenics would experience these symptoms constantly, and not just during psychotic episodes. Not sure what to make of that. Hmmmm. Maybe I should hypothesize that different parts of the cortex are “completely unmoored from each other” only during psychotic episodes, and the rest of the time they’re merely “mediocre at communicating”?
I am not sure that constant symptoms would be a necessary prediction of your theory: I could easily imagine that the out-of-sync regions with weak connections mostly learn to treat their connections as “mostly a bit of noise, little to gain here” and mostly ignore them during normal functioning (this also seems energetically efficient). But during exceptionally strong activation, they start using all channels and the neighbouring brain regions now need to make sense of the unusual input.

I cannot tell whether this story is more natural than a prediction of constant symptoms, but it does seem plausible to me.

Mart_Korz Jan 29, 2023, 9:38 PM
1 point
in reply to: alexnelja’s comment on: Scope Insensitivity
I do not know about scientific studies (which does not mean much), but at least anecdotally I think the answer is a yes at least for people who are not trained/experienced in making exactly these kinds of decisions.

One thing I have heard anecdotally is that people often significantly increase the prize when deciding to build/buy a house/car/vacation because they “are already spending lots of money, so who cares about adding 1% to the prize here and there to get neat extras” and thus spend years/months/days of income on things which they would not have bought if they had treated this as a separate decision.
This is a bit different from the bird-charity example, but it seems very related to me in that our intuitions have trouble with keeping track of absolute size.

Mart_Korz Dec 7, 2022, 6:31 PM
4 points
1
in reply to: Charlie Steiner’s comment on: [Link] Wavefunctions: from Linear Algebra to Spinors

Isn’t −1 inversion?

I think for quaternions, $(- 1) * q$ corresponds both to inversion and a 180 degree rotation.

When using quaternions to describe rotations in 3D space however, one can still represent rotations with unit-quaternions $q = cos (α / 2) + sin (α / 2) * n$ where n is a ‘unit vector’ distributed along the directions $i, j, k$ and indicates the rotation axis, and $α$ is the 3D rotation angle. If one wishes to rotate any orientation $x$ (same type of object as n) by q, the result is $q x q^{*}$ . Here, $q = - 1$ corresponds to $α / 2 = π$ and is thus a full 360 turn.

I have tried to read up on explanations for this a few times, but unfortunately never with full success. But usually people start talking about $q$ describing a “double cover” of the 3D rotations.

Maybe a bit of intuition about this relation can come from thinking about measured quantities in quantum mechanics as ‘expectation values’ of some operator $X$ written as $ψ^{†} X ψ$ : Here it becomes more intuitive that replacing $X = q^{*} q X q^{*} q$ (rotating the measured quantity back and forth by $α$ around the axis $n$ ) results in $ψ^{†} q^{*} q X q^{*} q ψ$ , which is an $α$ -rotated X measured on an $α / 2$ -rotated wavefunction.

Mart_Korz Nov 6, 2022, 9:57 PM
3 points
1
on: Eudaimonia, Voltron, and The Multi-Dimensional Crisis Revolution
Thank you.

I really like your framing of home—it seems very close to how John Vervaeke describes it, but somehow your description made something click for me.

I wish to be annealed by this process.

I’d like to share a similar framing of a different concept: beauty. I struggled with what I should call beautiful for a while, as there seemed to be both some objectivity to it, but also loads of seemingly arbitrary subjectiveness which just didn’t let me feel comfortable with feeling something to be beautiful. All the criteria I could use to call something beautiful just seemed off. A frame which helped me re-conceptualize much of my thinking about it is:

Beautiful is that which contributes to me wishing to be a part of this world.

Of this framing, I really like how compatible it is with being agentic, and also that it emphasizes subjectiveness without any sense of arbitrariness.

Mart_Korz Sep 24, 2022, 1:12 PM
3 points
2
on: A game of mattering
I will have to try this, thanks for pointing to a mistake I have made in my previous attempts at scheduling tasks!

One aspect which I have the feeling is also important to you (and is important to me) is that the system also has some beauty to it. I guess this is mostly because using the system should feel more rewarding than the alternative of “happen to forget about it” so that it can become a habit.

I recently read (/listened to) the shard theory of human values and I think that its model of how people decide on actions and especially how hyperbolic discounting arises fits really well with your descriptions:

To sustain ongoing motivation, it is important to clearly feel the motivation/progress in the moment. Our motivational system does not have access to an independent world model which could tell it that the currently non-motivating task is actually something to care about and won’t endorse it if there is no in-the-moment expectation of relevant progress. As you describe, your approach feels more like “making the abstract knowledge of progress graspable in-the-moment to one’s motivational system” instead of “trying to trick it into doing ‘the right thing’ ”.

Mart_Korz Sep 2, 2022, 7:18 PM
1 point
0
on: An Introduction to Current Theories of Consciousness
Regarding the transporter:

Why does “the copy is the same consciousness” imply that killing it is okay?

From these theories of consciousness, I do not see why the following would be ruled out:
- Killing a copy is equally bad as killing “the sole instance”
- It fully depends on the will of the person

Mart_Korz Sep 1, 2022, 5:51 AM
4 points
−1
in reply to: tgb’s comment on: An Introduction to Current Theories of Consciousness
oh.., right—it seems I actually drew B instead of C2. Here is the corrected C2 diagram:

Mart_Korz Aug 31, 2022, 8:35 PM
3 points
0
in reply to: Mart_Korz’s comment on: An Introduction to Current Theories of Consciousness
Okay, I think I managed to make at least the case C1-C2 intuitive with a Venn-type drawing:
(edit: originally did not use spades for C1)
The left half is C1, the right one is C2. In C1 we actually exclude both some winning ‘worlds’ and some losing worlds, while C2 only excludes losing worlds.
However due to symmetry reasons that I find hard to describe in words, but which are obvious in the diagrams, C1 is clearly advantageous and has a much better winning/loosing ratio.

(note that the ‘true’ Venn diagram would need to be higher dimensional so that one can have e.g. aces of hearts and clubs without also having the other two. But thanks to the symmetry, the drawing should still lead to the right conclusions.)

Mart_Korz Aug 31, 2022, 8:19 PM
1 point
0
in reply to: tgb’s comment on: An Introduction to Current Theories of Consciousness
Thanks for the attempt at giving an intuition!

I am not sure I follow your reasoning:

Maybe the intuition here is a little clearer, since we can see that winning hands that contain an ace of spades are all reported by C1 but some are not reported by C2, while all losing hands that contain an ace of spades are reported by both C1 and C2 (since there’s only one ace for C2 to choose from)

If I am not mistaken, this would at first only say that “in the situations where I have the ace of spades, then being told C1 implies higher chances than being told C2”? Each time I try to go from this to C1 > C2, I get stuck in a mental knot. [Edited to add:] With the diagrams below, I think I now get it: If we are in C2 and are told “You have the ace of spades”, we do have the same grey/loosing area as in C1, but the winning worlds only had a random ¹⁄₂ to ¹⁄₄ (one over the number of actual aces) chance of telling us about the ace of spades. Thus we should correspondingly reduce the belief that we are in these winning worlds. I hope this is finally correct reasoning. [end of edit]

I can only find an intuitive argument why B≠C is possible: If we initially imagine to be with equal probability in any of the possible worlds, when we are told “your cards contain an ace” we can rule out a bunch of them. If we are instead told “your cards contain this ace”, we have learned something different, and also something more specific. From this perspective it seems quite plausible that C > B

Mart_Korz Aug 21, 2022, 11:12 AM
1 point
0
in reply to: Nathan Young’s comment on: Wanted: Notation for credal resilience
Though it’s unclear to me if confidence intervals suggest this notation already. If you had less chance of moving your interval, then it would already be a smaller interval, right?
Counterexample: if I estimate the size of a tree, I might come up with CI 80 % [5 m, 6 m] by eye-balling it and expect that some friend will do a stronger measurement tomorrow. In that case, CI 80 % [5 m, 6m] still seems fine even though I expect the estimate to narrow down soon.
If the tree is instead from some medieval painting, my CI 80 % [5 m, 6 m] could still be true while I do not expect any significant changes to this estimate.
I think that the credal resilience is mostly the expected ease of gaining (/loosing??) additional information so that it does provides additional info to the current estimates.
But there is something to your statement: If I expected that someone could convince me that the tree is actually 20 m tall, this should already be included in my intervals.

Mart_Korz Aug 21, 2022, 10:33 AM
1 point
1
on: Wanted: Notation for credal resilience
I like the idea of your proposal—communicating how solidified one’s credences are should be helpful for quickly communicating on new topics (although I could imagine that one has to be quite good at dealing with probabilities for this to actually provide extra information).

Regarding your particular proposal “CR [ <probability of change>, <min size of change>, <time spent on question> }” is unintuitive to me:
- In “80% CI [5,20]” the probability is denoted with %, while its “unit”-less in your notation
- In “80% CI [5,20]”, the braces [] indicate an interval, while it is more of a tuple in your notation
A reformulation might be
80% CI [5,20] CR [0.1, 0.5, 1 day] --> 80% CI [5,20] $\times$ 10 % CR $_{1 d}$ [0.5}
Things I dislike about this proposal:
- The “CR $_{1 d}$ ” complicates notation. Possibly one could agree on a default time interval such as 1 day and only write the time range explicitly if it differs? Alternatively, “1 day CR” or “CR_1d” might be usable
- the “[0.5]” still feels unintuitive as notation and is still not an interval. Maybe there is a related theoretically-motivated quantity which could be used? Possibly something like the ‘expected information gain’ can be translated into its influence on the [5,20] intervals (with some reasonable assumptions on the distribution)?
  - 80% CI [5,20] $⋇$ 0.5 @ 10 % CR $_{1 d}$ might be an alternative, with the “\divideontimes” symbol being the $\pm$ of multiplication and hinting at the possible modification of the interval [5, 20]. For latex-free notation, “[5, 20] x0.5″ might be a suitable simplified version.
Overall, I don’t yet have a good intuition for how to think about expected information gain (especially if it is the expectation of someone else).
Also, it would be nice if there was a theoretical argument that one of the given numbers is redundant—getting an impression of what all the 6 numbers mean exactly would take me sufficiently long that it is probably better to just have the whole sentence
My 80% confidence interval is 5-20. I think there’s 10% chance I’d change my upper or lower bound by more than 50% of the current value if I spent another ~day investigating this.
But this would of course be less of a problem with experience :)

Mart_Korz Aug 20, 2022, 10:07 PM
1 point
on: Project Intro: Selection Theorems for Modularity
After reading your sequence today, there is one additional hypothesis which came to my mind, which I would like to make the case for (note that my knowledge about ML is very limited, there is a good chance that I am only confused):
Claim: Noise favours modular solutions compared to non-modular ones.
What makes me think that? You mention in Ten experiments that “We have some theories that predict modular solutions for tasks to be on average broader in the loss function landscape than non-modular solutions” and propose to experimentally test this.
If this is a true property of a whole model, then it will also (typically) be the case for modules on all size-scales. Plausibly the presence of noise creates an incentive to factor the problem into sub-problems which can then by solved with modules in a more noise-robust fashion (=broader optimum of solution). As this holds true on all size-scales, we create a bias towards modularity (among other things).
Is this different from the mentioned proposed modularity drivers? I think so. In experiment 8 you do mention input noise, which made me think about my hypothesis. But I think that ‘local’ noise might provide a push towards modules in all parts of the model via the above mechanism, which seems different to input noise.
Some further thoughts
- even if this effect is true, I have no idea about how strong it is
- noisy neurons actually seem a bit related to connection costs to me in that (for a suitable type of noise) recieving information from many inputs could become costly
- even if true, it might not make it easier to actually train modular models. This effect should mostly “punish modular solutions less than non-modular ones” instead of actually helping in training. A quick online search for “noisy neural network” indicated that these have indeed been researched and that performance does degrade. My first click mentioned the degrading performance and aimed to minimize the degradation. However I did not see non-biological results after adding “modularity” to the search (didn’t try for long, though).
- this is now pure speculation, but when reading “large models tend towards modularity”, I wondered whether there is some relation to noise? Could something like the finite bit resolution of weights lead to an effective noise that becomes significant at sufficient model sizes? (the answer might well be an obvious no)

Mart_Korz Aug 8, 2022, 3:57 PM
1 point
0
in reply to: Tom Paine’s comment on: What job should I do?
I just found the link for their summary on job-satisfaction in particular: https://80000hours.org/career-guide/job-satisfaction/

Mart_Korz Aug 5, 2022, 7:26 AM
3 points
in reply to: Alex_Altair’s comment on: Direct Observation
A re-interpretation that makes me perceive the lines as equal length without any problems is to label them as “some kind of magnets that hover above each other”

As this is somewhat realistic, while letting the lines integrate well with the rest of the railroad (maybe this is an artistic impression for Maglev trains?), this does not break immersion for me.

Mart_Korz Aug 1, 2022, 5:30 PM
1 point
in reply to: Valentin2026’s comment on: Can we model technological singularity as the phase transition?
Trends of different quantities:
Generally, I agree with your points :)
I recently stumbled upon this paper “The World’s Technological Capacity to Store, Communicate, and Compute Information”, which has some neat overviews regarding data storage, broadcasting and compute trends:
From a quick look at the Figures my impression is that compute and storage look very much like ‘just’ exponential, while there is a super-exponential figure (Fig. 4) for the total communication bandwidth (1986-2007)^[1]
General
I can see two mechanisms that potentially may make it less random.
That makes sense. Now that I think about this, I could well imagine that something like “scale is all you need” is sufficiently true that randomness doesn’t shift the expected date by a large amount.
if the data will predict for example singularity in 2030, it will likely mean AGI earlier than that.
Good point! I think that the time span around and before the first AGI will be most relevant to us as it probably provides the largest possibility to steer the outcome to something good, but this indeed is not the date we found get in a power-law singularity.
This feels quite related to the discussion around biological anchors for estimating the necessary compute for transformative AI and the conclusions one can draw from this. I feel that if one thinks that these are informative, even ‘just’ the exponential compute trends provide rather strong bounds (at least compared to having biological time-scales or such as reference).
t is not very likely that the control parameter will be that easily measurable and will obey power-law that good. I think it is a very high risk—very high gain project (very high gain, because if the prediction will be very clear it will be possible to persuade more people that the problem is important).
Regarding persuading people: I am not sure whether such a trend would make such a large psychological difference compared to the things that we already have: All Possible Views About Humanity’s Future Are Wild. But it would still be a noteworthy finding in any case
1. ^
  Quick hand-wavy estimate whether the trend continued in the last 15 years: If I just assume ‘trend continues’ to mean ‘doubling time halves every 7 years with a factor x40 from 2000 to 2007’ (this isn’t power law, but much easier for me to think about and hopefully close enough in this parameter range) we’d have to find an increase in global bandwidth by (a factor of x40 is roughly 1.5 OOM and we get this factor twice in the first 7 years and 4 times in the second step which makes) roughly 9 orders of magnitude from 2007 to 2021. I did not try to find current numbers, but 9 OOM sound unlikely to me. At least my data usage did increase by maybe 3 to 5 OOM, but 9 OOM just seems too high. Thus, I anecdotally conclude that this trend very probably slowed down in the last 15 years compared to ‘doubling time halves every 7 years’. But to be fair, I would not have correctly predicted the 1986-2007 trend either—so this shouldn’t be taken too seriously.

Mart_Korz Aug 1, 2022, 11:34 AM
4 points
0
on: What job should I do?
This does not quite match your question, but I want to recommend taking a look at https://80000hours.org/ if you don’t already know them.

Their focus is on providing resources for impactful altruistic careers, but they still have lots of nice general advice regarding job satisfaction and how to approach the topic of career choice. There also are examples where people describe their experiences with different career paths or the content of the paths themselves and lots more.

Depending how much you are interested in an altruistic focus for your career, you’ll find a larger or smaller portion of their writing relevant, but there is a lot of good stuff in any case :)

Mart_Korz Jul 12, 2022, 10:04 PM
4 points
on: Can we model technological singularity as the phase transition?
Compute: A very simple attempt at estimating (non-biological) computing power:
- A version of Moore’s law applies to the cost of computation power. And Moore’s law held true quite steadily, so we can assume an exponential growth,
- The figure in this article on the growth of the semiconductor industry shows significant oscillations in the growth rate in the last 25 years, but seems totally compatible with exponential growth plus a lot of noise.
If we just naively combine these two, we get two multiplying exponential growths for the total semiconductor compute over time, which again gives us exponential growth. This can be considered explosive in its own way but it is not a singularity.
Data storage: A quick search for global data storage lead me to this whitepaper where Fig. 1 on page 6 shows the “Annual Size of the Global Datasphere” and looks as if they just directly plotted an exponential function and did not directly use yearly empirical data. I am not sure whether the true data was just this close to an exponential or whether they did just decide to directly plot an exponential. An both cases, this speaks against super-exponential growth.
General remarks: I have the feeling that in technological singularity there are so many moving parts that even if we found a quantity which consistently shows power-law growth, we would still have a large uncertainty about the timing of singularity. For example, I assume that the transition from DNA as basically the only storage medium to human culture with written text was a significant game-changer. But the timing of this transition could have easily been different by a few centuries (certainly) or millennia (probably). Similarly I would expect that the first general artificial intelligence will be really relevant for a singularity, while it could easily be delayed by a few years /decades even if we know an expected date for a phase transition.
Possibly this is similar to supercooling of water, where it is the presence of ample condensation points which allows us to get the usual freezing temperature. If the liquid water is already beyond the standard freezing point, the first nucleus will decide whether the whole thing freezes.
Also, I could imagine “the amount of information that is usefully integrated in a decision-making agent” to be a quantity that is quite substrate-independent (DNA, people, humanity, AI models) so that one could try to find a trend over long time scales. But I expect that it will be super hard to define “amount of information” sufficiently well. At least “parameter counts in machine learning” (if we simply assume that a model will ~~not~~ integrate roughly as much data as it has parameters) does seem compatible with super-exponential growth.

Mart_Korz Jun 1, 2022, 9:20 PM
9 points
on: What will happen when an all-reaching AGI starts attempting to fix human character flaws?
If one believes the orthogonality thesis (and we only need a very weak version of it), just knowing that there is an AGI trying to improve the world is not enough to predict how exactly it would reason about the more quirky aspects about human character and values. It seems to me that something that could be called “AGI-humans” is quite possible, but a more alien-to-us “total hedonistic utility maximizing AGI” also seems possible.

From how I understood arguments of Eliezer Yudkowsky here, the way that we are selecting for AI models will favour models with consequentialist decision making (we do select the models that give good results), which tends towards the latter.
Because of this, I would expect an AGI to be more on the far-reaching/utilitarian end of affecting our lives.
With regards to
[...] accept some of the human character flaws and limitations, or will it strip it all away at the risk of hurting the human until the singularity of what is considered acceptable is achieved
if we are talking about an AGI that is aiming for good in a sufficiently aligned sense, it is not obvious that a significant “risk of hurting the human” is necessary to reach a value-optimal state.
But of course a utilitarian-leaning AGI will be more willing to risk actively doing harm if it thinks that the total expected outcome is improved.