Max Kaye

Karma: 12

Max Kaye Sep 19, 2020, 3:41 PM
1 point
in reply to: habryka’s comment on: Open & Welcome Thread—September 2020

Yeah, almost everyone who we ban who has any real content on the site is warned. It didn’t feel necessary for curi, because he has already received so much feedback about his activity on the site over the years (from many users as well as mods), and I saw very little probability of things changing because of a warning.

I think you’re denying him an important chance to do error correction via that decision. (This is a particularly important concept in CR/FI)

curi evidently wanted to change some things about his behaviour, otherwise he wouldn’t have updated his commenting policy. How do you know he wouldn’t have updated it more if you’d warned him? That’s exactly the type of criticism we (CR/FI) think is useful.

That sort of update is exactly the type of thing that would be reasonable to expect next time he came back (considering that he was away for 2 weeks when the ban was announced). He didn’t want to be banned, and he didn’t want to have shitty discussions, either. (I don’t know those things for certain, but I have high confidence.)

What probability would you assign to him continuing just as before if you said something like “If you keep continuing what you’re doing, I will ban you. It’s for these reasons.” Ideally, you could add “Here they are in the rules/faq/whatever”.

Practically, the chance of him changing is lower now because there isn’t any point if he’s never given any chances. So in some ways you were exactly right to think there’s low probability of him changing, it’s just that it was due to your actions. Actions which don’t need to be permanent, might I add.

Max Kaye Sep 19, 2020, 2:08 PM
−4 points
in reply to: lsusr’s comment on: Open & Welcome Thread—September 2020

I do not think the core disagreement between you and me comes from a failure of me to explain my thoughts clearly enough.

I don’t either.

The same goes for your position. The many words you have already written have failed to move me. I do not expect even more words to change this pattern.

Sure, we can stop.

Curi is being banned for wasting time with long, unproductive conversations.

I don’t know anywhere I could go to find out that this is a bannable offense. If it is not in a body of rules somewhere, then it should be added. If the mods are unwilling to add it to the rules, he should be unbanned, simple as that.

Maybe that idea is worth discussing? I think it’s reasonable. If something is an offense it should be publicly stated as such and new and continuing users should be able to point to it and say “that’s why”. It shouldn’t feel like it was made up on the fly as a special case—it’s a problem when new rules are invented ad-hoc and not canonicalized (I don’t have a problem with JIT rulebooks, it’s practical).

Max Kaye Sep 19, 2020, 7:11 AM
−5 points
in reply to: Kaj_Sotala’s comment on: Open & Welcome Thread—September 2020
The traditional guidance for up/downvotes has been “upvote what you would like want to see more of, downvote what you would like to see less of”. If this is how votes are interpreted, then heavy downvotes imply “the forum’s users would on average prefer to see less content of this kind”.

You’re using quotes but I am not sure what you’re quoting, do you just mean to emphasize/offset those clauses?

but people also have the right to choose who they want to spend their time with,

Sure, that might be part of the reason curi hadn’t been active on LW for 13 days at the time of the ban.

(continued)

even if someone who they preferred not to spend time with viewed that as being punished.

I don’t know if curi think’s it’s punishment. I think it’s punishment, and I think most ppl would agree that ‘A ban’ would be an answer to the question (in online forum contexts, generally) ‘What is an appropriate punishment?’ That would mean a ban is a punishment.

LW mods can do what they want; in essence it’s their site. I’m arguing:
1. it’s unnecessary
2. it was done improperly
3. it reflects badly on LW and creates a hostile culture to opposing ideas
4. (3) is antithetical to the opening lines of the LessWrong FAQ (which I quote below). Note: I’m introducing this argument in this post, I didn’t mention it originally.
5. significant parts of habryka’s post were factually incorrect. It was noted, btw, in FI that a) habryka’s comments were libel, and b) that curi’s reaction—quoted below—is mild and undercuts habryka’s claim.
curi wrote (in his post on the LW ban)

Those three things in combination, a propensity for long unproductive discussions, a history of threats against people who engage with him, and being the historically most downvoted account in LessWrong history, make me overall think it’s better for curi to find other places as potential discussion venues.

I didn’t threaten anyone. I’m guessing it was a careless wording. I think habryka should retract or clarify it. Above habryka used “attack[]” as a synonym for criticize. I don’t like that but it’s pretty standard language. But I don’t think using “threat[en]” as a synonym for criticize is reasonable.

“threaten” has meanings like “state one’s intention to take hostile action against someone in retribution for something done or not done” and “express one’s intention to harm or kill“ (New Oxford Dictionary). This is the one thing in the post that I strongly object to.

from the FI discussion:

JustinCEO: i think curi’s response to this libel is written in a super mild way

JustinCEO: which notably contrasts with being the sort of person who would have “a history of threats against people who engage with him” in the first place

LessWrong FAQ (original emphasis)

LessWrong is a community dedicated to improving our reasoning and decision-making. We seek to hold true beliefs and to be effective at accomplishing our goals. More generally, we want to develop and practice the art of human rationality.

To that end, LessWrong is a place to 1) develop and train rationality, and 2) apply one’s rationality to real-world problems.

I don’t think the things people have described (in this thread) as seemly important parts of LW are at all reflected by this quote, rather, they contradict it.

Max Kaye Sep 19, 2020, 6:57 AM
0 points
in reply to: habryka’s comment on: Open & Welcome Thread—September 2020

This is the definition that I had in mind when I wrote the notice above, sorry for any confusion it might have caused.

This definition doesn’t describe anything curi has done (see my sibling reply linked below), at least that I’ve seen. I’d appreciate any quotes you can provide.

https://www.lesswrong.com/posts/PkpuvsFYr6yuYnppy/open-and-welcome-thread-september-2020?commentId=H2tyDgoRFov8Xs8HS

Max Kaye Sep 19, 2020, 6:55 AM
−4 points
in reply to: lsusr’s comment on: Open & Welcome Thread—September 2020

define:threat

I prefer this definition, “a declaration of an intention or determination to inflict punishment, injury, etc., in retaliation for, or conditionally upon, some action or course; menace”.

This definition seems okay to me.

undue justice

I don’t know how justice can be undue, do you mean like undue or excessive prosecution? or persecution perhaps? thought I don’t think either prosecution or persecution describe anything curi’s done on LW. If you have counterexamples I would appreciate it if you could quote them.

We have substantial disagreements about what constitutes a threat,

Evidently yes, as do dictionaries.

I don’t think the dictionary definitions disagree much. It’s not a substantial disagreement. thesaurus.com seems to agree; it lists them as ~strong synonyms. the crux is retribution vs retaliation, and retaliation is more general. The mafia can threaten shopkeeps with violence if they don’t pay protection. I think retaliation is a better fitting word.

However, this still does not apply to anything curi has done!
What links here?
- Max Kaye's comment on Open & Welcome Thread—September 2020 by habryka (Sep 19, 2020, 6:57 AM; 0 points)

Max Kaye Sep 18, 2020, 12:53 PM
3 points
in reply to: lsusr’s comment on: Open & Welcome Thread—September 2020
lsusr said:

(1) Curi was warned at least once.

I’m reasonably sure the slack comments refers to events 3 years ago, not anything in the last few months. I’ll check, though.

There are some other comments about recent discussion in that thread, like this: https://www.lesswrong.com/posts/iAnXcZ5aGZzNc2J8L/the-law-of-least-effort-contributes-to-the-conjunction?commentId=38FzXA6g54ZKs3HQY

gjm said:

I had not looked, at that point; I took “mirrored” to mean taking copies of whole discussions, which would imply copying other people’s writing en masse. I have looked, now. I agree that what you’ve put there so far is probably OK both legally and morally.

My apologies for being a bit twitchy on this point; I should maybe explain for the benefit of other readers that the last time curi came to LW, he did take a whole pile of discussion from the LW slack and copy it en masse to the publicly-visible internet, which is one reason why I thought it plausible he might have done the same this time.

I don’t think there is case for (1). Unless gjm is a mod and there are things I don’t know?

lsusr said:

(2) Curi is being banned for wasting time with long, unproductive conversations. An appeals process would produce another long, unproductive conversation.

habryka explicitly mentions curi changing his LW commenting policy to be ‘less demanding’. I can see the motivation for expedition, but the mods don’t have to speedrun it. I think it’s bad there wasn’t any communication beforehand.

lsusr said:

(3) Specific quotes are unnecessary. It blindingly obvious from a glance through curi’s profile and even curi’s response you linked to that curi is damaging to productive dialogue on Less Wrong.

I don’t think that’s the case. His net karma has increased, and judging him for content on his blog—not his content on LW—does not establish whether he was ‘damaging to productive dialogue on Less Wrong’.

His posts on less wrong have been contributions, for example, www.lesswrong.com/posts/tKcdTsMFkYjnFEQJo/can-social-dynamics-explain-conjunction-fallacy-experimental is a direct response to of EY’s posts and it was net-upvoted. He followed that up with two more net-upvoted posts:
- www.lesswrong.com/posts/HpiTacu2P6c22GEzF/asch-conformity-could-explain-the-conjunction-fallacy
- www.lesswrong.com/posts/tKcdTsMFkYjnFEQJo/can-social-dynamics-explain-conjunction-fallacy-experimental
This is not the track record of someone wanting to waste time. I know there are disagreements between LW and curi / FI. If that’s the main point of contention, and that’s why he’s being banned, then so be it. But he doesn’t deserve to mistreated and have baseless accusations thrown at him.

lsusr said:

The strongest claim against curi is “a history of threats against people who engage with him [curi]”. I was able to confirm this via a quickly glance through curi’s past behavior on this site. In this comment threatens to escalate a dialogue by mirroring it off of this website. By the standards of collaborative online dialogue, this constitutes a threat against someone who engaged with him.

We have substantial disagreements about what constitutes a threat, in that case. I think a threat needs to involve something like danger, or violence, or something like that. It’s not a ‘threat’ to copy public discussion under fair use for criticism and commentary.

I googled the definition, and these are the two (for define:threat)
- a statement of an intention to inflict pain, injury, damage, or other hostile action on someone in retribution for something done or not done.
- a person or thing likely to cause damage or danger.
Neither of these apply.

Max Kaye Sep 18, 2020, 11:21 AM
1 point
in reply to: ChristianKl’s comment on: Open & Welcome Thread—September 2020

The above post explicitely says that the ban isn’t a personal judgement of curi. It’s rather a question of whether it’s good or not to have curi around on LessWrong and that’s where LW standards matter.

Isn’t it even worse then b/c no action was necessary?

But more to the point, isn’t the determination X person is not good to have around a personal judgement? It doesn’t apply to everyone else.

I think what habryka meant was that he wasn’t making a personal judgement.

Max Kaye Sep 18, 2020, 9:16 AM
3 points
in reply to: Sherrinford’s comment on: Open & Welcome Thread—September 2020
I’m not sure about other cases, but in this case curi wasn’t warned. If you’re interested, he and I discuss the ban in the first 30 mins of this stream

Max Kaye Sep 18, 2020, 8:57 AM
3 points
in reply to: habryka’s comment on: Open & Welcome Thread—September 2020
FYI and FWIW curi has updated the post to remove emails and reword the opening paragraph.

http://curi.us/2215-fallible-ideas-post-mortems and http://curi.us/2215-fallible-ideas-post-mortems#18059

Max Kaye Sep 18, 2020, 8:50 AM
1 point
in reply to: Rafael Harth’s comment on: Open & Welcome Thread—September 2020

Arguably, if there is something truly wrong with the list, I should have an issue with it.

This is non-obvious. It seems like you are extrapolating from yourself to everyone else. In my model, how much you would mind being on such a list is largely determent by how much social anxiety you generally feel. I would very much mind being on that list, even if I felt like it was justified.

I think this is fair, and additionally I maybe shouldn’t have used the word “truly”; it’s a very laden word. I do think that, on the balance of probabilities, my case does reduce the likelihood of something being foundationally wrong with it, though. (Note: I’ve said this in, what I think, is a LW friendly way. I’d say it differently on FI.)

One thing I do think, though, is that people’s social anxiety does not make things in general right or wrong, but can be decisive wrt thinking about a single action.

Another thing to point out is anonymous participation in FI is okay, it’s reasonably easy to use an anonymous/pseudonymous email to start with. curi’s blog/forum hybrid also allows for anonymous posting. FI is very pro-free-speech.

Knowing the existence of the list (again, even if it were justified) would also make me uneasy to talk to curi.

I think that’s okay, curi isn’t trying to attract everyone as an audience, and FI isn’t designed to be a forum which makes people feel comfortable, as such. It has different goals from e.g. LW or a philosophy subreddit.

I think we’d agree that norms at FI aren’t typical and aren’t for everyone. It’s a place where anyone can post, but that doesn’t mean that everyone should, sorta thing.

Max Kaye Sep 18, 2020, 8:12 AM
13 points
in reply to: habryka’s comment on: Open & Welcome Thread—September 2020

Today we have banned two users, curi and Periergo from LessWrong for two years each.

I wanted to reply to this because I don’t think it’s right to judge curi the way you have. Periergo I don’t have an issue w/. (it’s a sockpuppet acct anyway)

I think your decision should not go unquestioned/uncriticized, which is why I’m posting. I also think you should reconsider curi’s ban under a sort of appeals process.

Also, the LW moderation process is evidently transparent enough for me to make this criticism, and that is notable and good. I am grateful for that.

On his blog, he and others maintain a long list of people who engaged with him and others in the Critical Rationalist community, but then stopped, in a way that is very hard to read as anything but a public attack.

You are judging curi and FI (Fallible Ideas) via your standards (LW standards), not FI’s standards. I think this is problematic.

I’d like to note I am on that list. (like ¹⁄₂ way down) I am also a public figure in Australia, having founded a federal political party based on epistemic principles with nearly 9k members. I am okay with being on that list. Arguably, if there is something truly wrong with the list, I should have an issue with it. I knew about being on that list earlier this year, before I returned to FI. Being on the list was not a factor in my decision.

There is nothing immoral or malicious about curi.us/2215. I can understand why you would find it distasteful, but that’s not a decisive reason to ban someone or condemn their actions.

A few hours ago, curi and I discussed elements about the ban and curi.us/2215 on his stream. I recommend watching a few minutes starting at 5:50 and at 19:00, for transparency you might also be interested in 23:40 → 24:00. (you can watch on 2x speed, should be fine)

Particularly, I discuss my presence on curi.us/2215 at 5:50

You say:

a long list of people who engaged with him and others in the Critical Rationalist community

There are 33 by my count (including me). The list spans a decade, and is there for a particular purpose, and it is not to publicly shame people in to returning, or to be mean for the sake of it. I’d like to point out some quotes from the first paragraph of curi.us/2215:

This is a list of ppl who had discussion contact with FI and then quit/evaded/lied/etc. It would be good to find patterns about what goes wrong. People who left are welcome to come back and try again.

Notably, you don’t end up on the list if you are active. Also, although it’s not explicitly mentioned in the top paragraph; a crucial thing is that those on the list have left and avoided discussion about it. Discussion is much more important in FI than most philosophy forums—it’s how we learn from each other, make sure we understand, offer criticism and assist with error correction. You’re not under any obligation to discuss something, but if you have criticisms and refuse to share them: you’re preventing error correction; and if you leave to evade criticism then you’re not living by your values and philosophy.

The people listed on curi.us/2215 have participated in a public philosophy forum for which there are established norms that are not typical and are different from LW. FI views the act of truth-seeking differently. While our (LW/FI) schools of thought disagree on epistemology, both schools have norms that are related to their epistemic ideas. Ours look different.

It is unfair to punish someone for an act done outside of your jurisdiction under different established norms. If curi were putting LW people on his list, or publishing off-topic stuff at LW, sure, take moderation action. None of those things happened. In fact, the main reason you’ve provided for even knowing about that list is via the sockpuppet you banned.

Sockpuppet accounts are not used to make the lives of their victims easier. By banning curi along with Periergo you have facilitated a (minor) victory for Periergo. This is not right.

a history of threats against people who engage with him

THIS IS A SERIOUS ALLEGATION! PLEASE PROVIDE QUOTES

curi prefers to discuss in public so they should be easy to find and verify. I have never known curi to threaten people. He may criticise them, but he does not threaten them.

Notably, curi has consistently and loudly opposed violence and the initiation of force, if people ask him to leave them alone (provided they haven’t e.g. committed a crime against him), he respects that.

being the historically most downvoted account in LessWrong history

This is not a reason to ban him, or anyone. Being disliked is not a reason for punishment.

Those three things in combination, a propensity for long unproductive discussions, a history of threats against people who engage with him, and being the historically most downvoted account in LessWrong history, make me overall think it’s better for curi to find other places as potential discussion venues.

“a history of threats against people who engage with him” has not been established or substantiated.

he seems well-intentioned

I believe he is. As far as I can tell he’s gone to great personal expense and trouble to keep FI alive for no other reason than that his sense of morality demands it. (That might be over simplifying things, but I think the essence is the same. I think he believes it is the right thing to do, and it is a necessary thing to do)

I do also think his most recent series of posts and comments is overall much less bad than the posts and comments he posted a few years ago (where most of his negative karma comes from)

He has gained karma since returning to LW briefly. I think you should retract the part about him having negative karma b/c it misrepresents the situation. He could have made a new account and he would have positive karma now. That means your judgement is based on past behaviour that was already punished. ~~This is double jeopardy.~~ (Edit: after some discussion on FI it looks like this isn’t double jeopardy, just double punishment. Double jeopardy specifically refers to being on trial for the same offense twice, not being punished twice.)

Moreover, curi is being punished for being honest and transparent. If he had registered a new account and hidden his identity, would you have banned him only based on his actions this past 1-2 months? If you can say yes, then fine, but I don’t think your argument holds in this case the only part that is verifiable is based on your disapproval of his discussion methods. Disagreeing with him is fine. I think a proportionate response would be a warning.

As it stands no warning was given, and no attempt to learn his plans was made. I think doing that would be proportionate and appropriate. A ban is not.

It is significant that curi is not able to discuss this ban himself. I am voluntarily doing this, of my own accord. He was not able to defend himself or provide explanation.

This is especially problematic as you specifically say you think he was improving compared with his conduct several years ago.

I do also think his most recent series of posts and comments is overall much less bad than the posts and comments he posted a few years ago (where most of his negative karma comes from), but they still don’t strike me as great contributions to the LessWrong canon

This alone is not enough. A warning is proportionate.

are all low-karma

Unpopularity is no reason for a ban

and I assign too high of a probability that old patterns will repeat themselves.

How is this different to pre-crime?

I think, given he had deliberately changed his modus operandi weeks ago and has not posted in 13 days, this is unfair and overly judgmental.

You go on to say:

and I do think that was the right move, but I don’t think it’s enough to tip the scales on this issue.

What could curi have done differently which would have tipped the scales? If there is no acceptable thing he could have done, why was action not taken weeks ago when he was active?

I believe it is fundamentally unjust to delay action in this fashion without talking with him first. curi has an incredibly long track record of discussion, he is very open to it. He is not someone who avoids taking responsibility for things; quite the opposite. If you had engaged him, I am confident he would have discussed things with you.

and to generally err on the side of curating our userbase pretty heavily and maintaining high standards.

It makes sense that you want to cultivate the best rational forums you can. I think that is a good goal. However, again, there were other, less extreme and more proportionate actions that could have been taken first, especially seeing as curi had changed his LW discussion policy and was inactive at the time of the ban.

We presumably disagree on the meaning of ‘high standards’, but I don’t think that’s particularly relevant here.

This means making difficult moderation decision long before it is proven “beyond a reasonable doubt” that someone is not a net-positive contributor to the site.

There were many alternative actions you could have taken. For example, a 1-month ban. Restricting curi to only posting on his own shortform. Warning him of the circumstances and consequences under conditions, etc.

In this case, I think it is definitely not proven beyond a reasonable doubt that curi is overall net-negative for the site

I’m glad you’ve mentioned this, but LW is not a court of law and you are not bound to those standards (and no punishment here is comparable to the punishment a court might distribute). I think there are other good reasons for reconsidering curi’s ban.

banning him might well be a mistake, but I think the probabilities weigh heavily enough in favor of the net-negative, and the worst-case outcomes are bad-enough, that on-net I think this is the right choice.

I think there is a critical point to be made here: you could have taken no action at this time and put a mod-notification for activity on his account. If he were to return and do something you deemed unacceptable, you could swiftly warn him. If he did it again, then a short-term ban. Instead, this is a sledge-sized banhammer used when other options were available. It is a decision that is now publicly on LW and indicates that LW is possibly intolerant of things other than irrationality. I don’t think this is reflective of LW, and I think it reflects poorly on the moderation policies here. I don’t think it needs to be that way, though.

I think a conditional unbanning (i.e. 1 warning, with the next action being a swift short ban) is an appropriate action for the moderation team to make, and I implore you to reconsider your decision.

If you think this is not appropriate, then I request you explain why 2 years is an appropriate length of time, and why Periergo and curi should have identical ban lengths.

The alternative to pacificity does not need to be so heavy handed.

I’d also like to note that curi has published a post on his blog regarding this ban; I read it after drafting this reply: http://curi.us/2381-less-wrong-banned-me

Max Kaye Sep 18, 2020, 2:20 AM
4 points
in reply to: habryka’s comment on: Open & Welcome Thread—September 2020
FYI I am on that list and fine with it—curi and I discussed this post a bit here: https://www.youtube.com/watch?v=MxVzxS8uMto

I think you’re wrong on multiple counts. Will reply more in a few hours.

Max Kaye Sep 15, 2020, 12:24 PM
2 points
in reply to: TurnTrout’s comment on: Max Kaye’s Shortform

\usepackage{cleveref}

Cool, thanks. I think I was missing \usepackage{cleveref}. I actually wrote the post in latex (the post for which I asked this question), but the lesswrong docs on using latex are lacking. for example they don’t tell you they support importing stuff and don’t list what is supported.

\Cref{eq:1} is an amazing new discovery; before Max Kaye, no one grasped the perfect and utter truth of \cref{eq:1}.

I use crefs in the .tex file linked above. I suppose I should have been more specific and asked “does anyone know how to label equations and reference them on lesswrong?” instead.

Max Kaye Sep 7, 2020, 12:55 AM
1 point
in reply to: interstice’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

I think that the argument about emulating one Turing machine with another is the best you’re going to get in full generality.

In that case I especially don’t think that argument answers the question in OP.

I’ve left some details in another reply about why I think the constant overhead argument is flawed.

So while SI and humans might have very different notions of simplicity at first, they will eventually come to have the same notion, after they see enough data from the world.

I don’t think this is true. I do agree some conclusions would be converged on by both systems (SI and humans), but I don’t think simplicity needs to be one of them.

If an emulation of a human takes X bits to specify, it means a human can beat SI at binary predictions at most X times(roughly) on a given task before SI wises up.

Uhh, I don’t follow this. Could you explain or link to an explanation please?

The quantity that matters is how many bits it takes to specify the mind, not store it(storage is free for SI just like computation time).

I don’t think that applies here. I think that data is part of the program.

For the human brain this shouldn’t be too much more than the length of the human genome, about 3.3 GB.

You would have to raise the program like a human child in that case^1. Can you really make the case you’re predicting something or creating new knowledge via SI if you have to spend (the equiv. of) 20 human years to get it to a useful state?

How would you ask multiple questions? Practically, you’d save the state and load that state in a new SI machine (or whatever). This means the data is part of the program.

Moreover, if you did have to raise the program like any other newborn, you have to use some non-SI process to create all the knowledge in that system (because people don’t use SI, or if they do use SI, they have other system(s) too).

1: at least in terms of knowledge; though if you used the complete human genome arguably you’d need to simulate a mother and other ppl too, but they have to be good simulations after the first few years, which is a regressive problem. So it’s probably easier to instantiate it in a body and raise it like a person b/c human people are already suitable. You also need to worry about it becoming mistaken (intuitively one disagrees with most people on most things we’d use an SI program for).

Max Kaye Sep 7, 2020, 12:44 AM
1 point
in reply to: johnswentworth’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

The solution to the “large overhead” problem is to amortize the cost of the human simulation over a large number of English sentences and predictions.

That seems a fair approach in general, like how can we use the program efficiently/profitably, but I don’t think it answers the question in OP. I think it actually actually implies the opposite effect: as you go through more layers of abstraction you get more and more complex (i.e. simplicity doesn’t hold across layers of abstraction). That’s why the strategy you mention needs to be over ever larger and larger problem spaces to make sense.

So this would still mean most of our reasoning about Occam’s Razor wouldn’t apply to SI.

A short English sentence then adds only a small amount of marginal complexity to the program—i.e. adding one more sentence (and corresponding predictions) only adds a short string to the program.

I’m not sure we (humanity) know enough to claim only a short string needs to be added. I think GPT-3 hints at a counter-example b/c GTP has been growing geometrically.

Moreover, I don’t think we have any programs or ideas for programs that are anywhere near sophisticated enough to answer meaningful Qs—unless they just regurgitate an answer. So we don’t have a good reason to claim to know what we’ll need to add to extend your solution to handle more and more cases (especially increasingly technical/sophisticated cases).

Intuitively I think there is (physically) a way to do something like what you describe efficiently because humans are an example of this—we have no known limit for understanding new ideas. However, it’s not okay to use this as a hypothetical SI program b/c such a program does other stuff we don’t know how to do with SI programs (like taking into account itself, other actors, and the universe broadly).

If the hypothetical program does stuff we don’t understand and we also don’t understand its data encoding methods, then I don’t think we can make claims about how much data we’d need to add.

I think it’s reasonable there would be no upper limit on the amount of data we’d need to add to such a program as we input increasingly sophisticated questions. I also think it’s intuitive there’s no upper limit on this data requirement (for both people and the hypothetical programs you mention).
What links here?
- Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond? by curi (Sep 4, 2020, 7:46 PM; 2 points)

Max Kaye Sep 5, 2020, 8:54 PM
1 point
in reply to: interstice’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

for any simple English hypothesis, we can convert it to code by running a simulation of a human and giving them the hypothesis as input, then asking them to predict what will happen next. Therefore the English and code-complexity can differ by at most a constant.

Some things are quick for people to do and some things are hard. Some ideas have had multiple people continuously arguing for centuries. I think this either means you can’t apply a simulation of a person like this, or some inputs have unbounded overhead.

Solomonoff induction is fine with inputs taking unboundedly long to run. There might be cases where the human doesn’t converge to a stable answer even after an indefinite amount of time. But if a “simple” hypothesis can have people debating indefinitely about what it actually predicts, I’m okay with saying that it’s not actually simple(or that it’s too vague to count as a hypothesis), so it’s okay if SI doesn’t return an answer in those cases.

Yeah okay, I think that’s fair.

My issue generally (which is in my reply to johnswentworth) is the overhead is non-negligible if you’re going to invoke a human. In that case we can’t conclude that simplicity would carry over from the english representation to the code representation. So this argument doesn’t answer the question.

You do say it’s a loose bound, but I don’t think it’s useful. One big reason is that the overhead would dwarf any program we’d ever run, and pretty much every program would look identical b/c of the overhead. For simplicity to carry over we need relatively small overhead (even like the entire python runtime is only like 20mb extra via py2exe, much smaller than a mind and definitely not simple).

Maybe it’s worth mentioning the question in OP: I read it as: “why would stuff the simplicity an idea had in one form (code) necessarily correspond to simplicity when it is in another form (english)? or more generally: why would the complexity of an idea stay roughly the same when the idea is expressed through different abstraction layers?” After that there’s implications for Occam’s Razor. Particularly it’s relevant b/c occam’s razor would give different answers when comparing ideas at different levels of abstraction, and if that’s the case we can’t be sure that ideas which are simple in english will be simple in code and we don’t have a reason for Occam’s Razor applying to SI.

Does that line up with what you think OP is about? If not we might be talking cross-purposes.

I mean you can pass functions as arguments to other functions and perform operations on them.

Ahh okay; first class functions.

Re “perform operations on [functions]”: you can make new functions and partially or fully apply functions, but that’s about it. (that does mean you can partially apply functions and pass them on, though, which is super useful)

So if you had one programming language with dictionaries built-in and other without, the one with dictionaries gets at most a constant advantage in code-length.

I agree with you that the theoretical upper bound on the minimum overhead is the size of a compiler/interpreter.

I think we might disagree on this, though: the compiler/interpreter includes data such as initial conditions (e.g. binary extensions, dynamic libraries, etc). I think this is an issue b/c there’s no upper bound on that. If you invoke a whole person then it’s an issue b/c for that person to solve more and more complex problems (or a wider and wider array) those initial conditions are going to grow correspondingly. Our estimates for the data requirements to store a mind are like $10^{20}$ bits. I’d expect the minimum required data to drop as problems got “simpler”, but my intuition is that pattern is not the same pattern as what Occam’s Razor gives us (e.g. minds taking less data can still think about what Thor would do).

Max Kaye Sep 5, 2020, 4:37 PM
1 point
in reply to: johnswentworth’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

the M1-simulator may be long, but its length is completely independent what we’re predicting—thus, the M2-Kolmogorov-complexity of a string is at most the M1-Kolmogorov-complexity plus a constant (where the constant is the length of the M1-simulator program).

I agree with this, but I don’t think it answers the question. (i.e. it’s not a relevant argument^([1]))

Given the English sentence, the simulated human should then be able to predict anything a physical human could predict given the same English sentence.

There’s a large edge case where the overhead constant is ~greater than the program. in those cases it’s not the case that simplicity transitions across layers of abstraction.

That edge case means this doesn’t follow:

Thus, if something has a short English description, then there exists a short (up to a constant) code description

[1]: Edit: it could be relevant but not the whole story; but in that case it’s missing a sizable chunk.

Max Kaye Sep 5, 2020, 3:48 PM
1 point
in reply to: Pongo’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

On the literature that addresses your question: here is a classic LW post on this sort of question.

The linked post doesn’t seem to answer it, e.g. in the 4th paragraph EY says:

Why, exactly, is the length of an English sentence a poor measure of complexity? Because when you speak a sentence aloud, you are using labels for concepts that the listener shares—the receiver has already stored the complexity in them.

I also don’t think it fully addresses the question—or even partially in a useful way, e.g. EY says:

It’s enormously easier (as it turns out) to write a computer program that simulates Maxwell’s equations, compared to a computer program that simulates an intelligent emotional mind like Thor.

The formalism of Solomonoff induction measures the “complexity of a description” by the length of the shortest computer program which produces that description as an output.

But this bakes in knowledge about measuring stuff. Maxwell’s equations are—in part—easier to code because we have a way to describe measurements that’s easy to compute. That representation is via an abstraction layer! It uses labels for concepts too.

Max Kaye Sep 5, 2020, 3:17 PM
1 point
in reply to: interstice’s comment on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?

One somewhat silly reason: for any simple English hypothesis, we can convert it to code by running a simulation of a human and giving them the hypothesis as input, then asking them to predict what will happen next.

Some things are quick for people to do and some things are hard. Some ideas have had multiple people continuously arguing for centuries. I think this either means you can’t apply a simulation of a person like this, or some inputs have unbounded overhead.

because coding languages were designed to be understandable by humans and have syntax similar to human languages.

You should include all levels of abstraction in your reasoning, like raw bytecode. It’s both low level and can be written by humans. It’s not necessarily fun but it’s possible. What about things people design at a transistor level?

e.g. functional programming languages allow you to treat functions as objects.

I use Haskell and have no idea what you’re talking about.

I think abstraction in programming is different to what you mean; e.g. dictionary might a simple data structure to use, but a list of tuples is harder to use even though you can implement a dictionary using a list of tuples. the abstraction layer (the implementation) is what makes complex operations on list of tuples into simple operations on dictionaries.

Max Kaye Sep 5, 2020, 3:06 PM
1 point
on: Why would code/English or low-abstraction/high-abstraction simplicity or brevity correspond?
Brevity of code and english can correspond via abstraction.

I don’t know why brevity in low and high abstraction programs/explanations/ideas would correspond (I suspect they wouldn’t). If brevity in low/high abstraction stuff corresponded; isn’t that like contradictory? If a simple explanation in high abstraction is also simple in low abstraction then abstraction feels broken; typically ideas only become simple after abstraction. Put another way: the reason to use abstraction is to make ideas/thing that are highly complex into things that are less complex.

I think Occam’s Razor makes sense only if you take into account abstractions (note: O.R. itself is still a rule of thumb regardless). Occam’s Razor doesn’t make sense if you think about all the extra stuff an explanation invokes—partially because that body of knowledge grows as we learn more, and good ideas become more consistent with the population of other ideas over time.

When people think of short code they think of doing complex stuff with a few lines of code. e.g. cat asdf.log | cut -d ',' -f 3 | sort | uniq. When people think of (good) short ideas they think of ideas which are made of a few well-established concepts that are widely accessible and easy to talk about. e.g. we have seasons because energy from sunlight fluctuates ~sinusoidally through our annual orbit.

One of the ways SI can use abstraction is via the abstraction being encoded in both the program, program inputs, and the observation data.

(I think) SI uses an arbitrary alphabet of instructions (for both programs and data), so you can design particular abstractions into your SI instruction/data language. Of course the program would be a bit useless for any other problem than the one you designed it for, in this case.

Is there literature arguing that code and English brevity usually or always correspond to each other?

I don’t know of any.

If not, then most of our reasons for accepting Occam’s Razor wouldn’t apply to SI.

I think some of the reasoning makes sense in a pointless sort of way. e.g. the hypothesis 1100 corresponds to the program “output 1 and stop”. The input data is from an experiment, and the experiment was “does the observation match our theory?”, and the result was 1. The program 1100 gets fed into SI pretty early, and it matches the predicted output. The reason this works is that SI found a program which has info about ‘the observation matching the theory’ already encoded, and we fed in observation data with that encoding. Similarly, the question “does the observation match our theory?” is short and elegant like the program. The whole thing works out because all the real work is done elsewhere (in the abstraction layer).