lincolnquirk

• lincolnquirkFeb 22, 2023, 3:05 PM

  9 points

  3

  on: lin­coln­quirk’s Shortform

  Tim Urban’s new book, What’s Our Problem, is out as of yesterday. I’ve started reading it and it’s good so far, and very applicable to rationality training. waitbutwhy.com

lin­coln­quirk’s Shortform

• lincolnquirkNov 24, 2022, 1:58 PM

  3 points

  0

  on: LW Beta Fea­ture: Side-Comments

  Excited about this!

  Points of feedback:

  1. I don’t like to have to scroll my screen horizontally to read the comment. (I notice there’s a lot of perfectly good unused white space on the left side; comments would probably fit horizontally if you pushed everything to the left!)

  2. Sometimes when you mouse over the side-comment icon, it tries to scroll the page to make the comment readable. This is very surprising and makes me lose my place.

  3. Hovering over the icon makes the comment appear briefly. If I then want to scroll in order to read the comment, there seems to be no way to ‘stay hovered’—I have to click and toggle it, to make the comment stick around so I can actually read it. (This plus being forced to scroll the screen makes the hover feature kind of useless.)

  Overall, feeling optimistic though, and will probably use this.

• lincolnquirkOct 11, 2022, 1:15 PM

  2 points

  0

  in reply to: cveres’s comment on: Don’t ex­pect AGI any­time soon

  I think your argument is wrong, but interestingly so. I think DL is probably doing symbolic reasoning of a sort, and it sounds like you think it is not (because it makes errors?)

  Do you think humans do symbolic reasoning? If so, why do humans make errors? Why do you think a DL system won’t be able to eventually correct its errors in the same way humans do?

  My hypothesis is that DL systems are doing a sort of fuzzy finite-depth symbolic reasoning—it has capacity to understand the productions at a surface level and can apply them (subject to contextual clues, in an error-prone way) step by step, but once you ask for sufficient depth it will get confused and fail. Unlike humans, feedforward neural nets can’t think for longer and churn step by step yet; but if someone were to figure out a way to build a looping option into the architecture then I won’t be surprised to see DL systems which can go a lot further on symbolic reasoning than they currently do.

• lincolnquirkOct 4, 2022, 7:31 PM

  2 points

  0

  on: Will you let your kid play foot­ball?

  What is Pop Warner in this context? I have googled it and it sounds like he was one of the founders of modern American football, but I don’t understand what it is in contrast to. Is there some other (presumably safer) ruleset?

• lincolnquirkSep 10, 2022, 1:07 AM

  37 points

  17

  in reply to: habryka’s comment on: My emo­tional re­ac­tion to the cur­rent fund­ing situation

  (Inside-of-door-posted hotel room prices are called “rack rates” and nobody actually pays those. This is definitely a miscommunication.)

• lincolnquirkAug 19, 2022, 12:39 AM

  8 points

  7

  on: What does moral progress con­sist of?

  I am guilty of being a zero-to-one, rather than one-to-many, type person. It seems far easier and more interesting to me, to create new forms of progress of any sort, rather than convincing people to adopt better ideas.

  I guess the project of convincing people seems hard? Like, if I come up with something awesome that’s new, it seems easier to get it into people’s hands, rather than taking an existing thing which people have already rejected and telling them “hey this is actually cool, let’s look again”.

  All that said, I do find this idea-space intriguing partly thanks to this post—it makes me want to think of ways of doing more one-to-many type stuff. I’ve been recently drawn into living in DC and I think the DC effective altruism folks are much more on the one-to-many side of the world.

• lincolnquirkAug 9, 2022, 9:00 PM

  12 points

  4

  on: Pro­posal: Con­sider not us­ing dis­tance-di­rec­tion-di­men­sion words in ab­stract discussions

  Upvoted for raising something to conscious attention, that I have never previously considered might be worth paying attention to.

  (Slightly grumpy that I’m now going to have a new form of cognitive overhead probably 10+ times per day… these are the risks we take reading LW :P)

• lincolnquirkJul 26, 2022, 2:33 AM

  5 points

  2

  on: How Promis­ing is The­o­ret­i­cal Re­search on Ra­tion­al­ity? Seek­ing Ca­reer Advice

  Look, I don’t know you at all. So please do ignore me if what I’m saying doesn’t seem right, or just if you want to, or whatever.

  I’m a bit worried that you’re seeking approval, not advice? If this is so, know that I for one approve of your chosen path. You are allowed to spend a few years focusing on things that you are passionate about, which (if it works out) may result in you being happy and productive and possibly making the world better.

  If you are in fact seeking advice, you should explain what your goal is. If your goal is to make the maximum impact possible — it’s worth at least hundreds of hours trying to see if you can learn more & motivate yourself along a path which seems like it combines high impact with personal resonance. I wouldn’t discount philosophy along this angle, but (for example) it sounds like you may not know that much about the potential of policy careers; there are plenty that do not require particularly strong mathematical skills (… or even any particularly difficult skills beyond some basic extraversion, resistance to boredom and willingness to spend literal decades grinding away within bureaucracies).

  If your goal is to be happy, I think you will be happy doing philosophy, and I think you have a potential to make a huge impact that way. Certainly there are a decent number of full-time philosophers within effective altruism who I have huge respect for (Macaskill, Ord, Bostrom, Greaves, and Trammell jump to mind). Plus, you can save a few hundred hours, which seems pretty important if you might already know the outcome of your experimentation!

• lincolnquirkJul 18, 2022, 2:54 AM

  2 points

  0

  in reply to: Quintin Pope’s comment on: Why I Think Abrupt AI Takeoff

  Thanks! This is very helpful, and yes, I did mean to refer to grokking! Will update the post.

Why I Think Abrupt AI Takeoff

• lincolnquirkJun 28, 2022, 9:18 PM

  17 points

  7

  on: What Are You Track­ing In Your Head?

  Nice post!

  One of my fears is that the True List is super long, because most things-being-tracked are products of expertise in a particular field and there are just so many different fields.

  Nevertheless:

  • In product/​ux design, tracking the way things will seem to a naive user who has never seen the product before.

  • In navigation, tracking which way north is.

  • I have a ton of “tracking” habits when writing code:

    • types of variables (and simulated-in-my-head values for such)

    • refactors that want to be done but don’t quite have enough impetus for yet

    • loose ends, such as allocated-but-not-freed resources, or false symmetry (something that looks like it should be symmetric but isn’t in some critical way), or other potentially-misleading things that need to be explained

    • [there are probably a lot more of these that I am not going to write down now]

• lincolnquirkJun 24, 2022, 6:30 PM

  5 points

  1

  on: Let’s See You Write That Cor­rigi­bil­ity Tag

  Here’s my attempt. I haven’t read any of the other comments or the tag yet. I probably spent ~60-90m total on this, spread across a few days.

  On kill switches

  • low impact somehow but I don’t know how

  • Go slow enough so that people can see what you’re doing

  • Have a bunch of “safewords” and other kill-switches installed at different places, some hopefully hard-to-reach by the AI. Test them regularly, and consider it a deadly flaw if one stops working.

  On the AI accurately knowing what it is doing, and pointing at things in the real world

  • watch all the metrics (!)

  • Predict all the metrics you watch, and ask humans about any anomalous metrics that you are watching

  • group inputs and outputs separately and treat inputs as sacred. Perhaps by having an epistemic module which is incentivized by producing true predictions about the world. Make the epistemic module the one that gets all the juice. Planning module should not have any way to influence it.

  On responding predictably

  • Require inputs of some kind to get more outputs (something about control theory?) - like a power steering wheel, or an ebike, it helps the human by amplifying the motions you make but not actually Doing Anything on its own

  • Have metrics look smooth on an input/​output response curve. No sharp edges. let the humans be the one to turn the knobs.

  • Try things. Try things on a small scale before going big. (define “big”? maybe look at the metrics!)

  On epistemology and ontology

  • Ask for feedback both about things you tried; and experiments you want to run

  • make the epistemic module have a stable ontology. If it wants to change the ontology, it needs to create a new epistemic module, and leave the old one running. The new module has to be able not only to predict physical things about the world but also what the old model would say about things being pointed-to and tagged as important in the old models ontology.

  • have a concept of goal metrics and side effects. Try to predict what the human will say about which metrics are goals and which are side effects. (hmm..this is incoherent because you can come up with many irrelevant metrics… what even is a relevant metric. Maybe the system should be grouping metrics using some kind of natural unsupervised clustering system and then trying to predict the prioritization of metric groups? And then to explain the groups it gives samples of the actual thing it’s measuring. Still difficult to make sense of this. )

  On planning

  • Bound or penalize plans in complexity and time.

    • “complexity” is hard to define here, and isn’t Kolmogorov I don’t think.

  • Harshly penalize plans which lean on predicting what other agents will do. Ban plans which depend on prediction of what your operator will do. (AI can still predict what the operator will think of the plan, but that prediction should not feed into any plan itself.)

  • Separate planning from other parts of AI cognition, such that you are never confused about whether you are making a plan or doing other things, such as predicting things about the world, or executing a plan.

  • Don’t execute plans at all

  • Train the AI to identify the purpose(s) of each step in the plan. Give negative feedback when steps lead to outcomes that are not identified ahead of time.

  On transparency

  • A part of the AI should be trained to explicate what the rest of the AI is “doing”/​focused on at all times. Find a way to make this into a feedback cycle.

  • AI should be arranged so it is mostly only doing one thing at a time, and that doesn’t switch its focus too many times each minute.

• lincolnquirkJun 13, 2022, 6:13 PM

  12 points

  in reply to: SarahNibs’s comment on: On A List of Lethalities

  I notice that I am extremely surprised by your internship training. Its existence, its lessons and the impact it had on you (not you specifically, just a person who didn’t come in with that mindset) are all things I don’t think I would have predicted. I would be thrilled if you would write as much as you can bring yourself to about this, braindump format is fine, into a top level post!

• lincolnquirkJun 10, 2022, 10:58 PM

  5 points

  on: Leav­ing Google, Join­ing the Nu­cleic Acid Observatory

  Congrats, I’m excited about this!

• lincolnquirkMar 28, 2022, 3:35 AM

  15 points

  on: Con­se­quen­tial­ist ve­g­anism

  I’ve been turning this over in my head for a while now. (Currently eating mostly vegan fwiw, but I am not sure if this is the right decision.)

  I think the main argument against veganism is that it actually incurs quite a large cost. Being vegan is a massive lifestyle change with ripple effects that extend into one’s social life. This argument falls under your “there are higher-impact uses of your (time/​energy/​money/​etc.)”, but what you wrote doesn’t capture the reasons why this is important.

  most of us do not have good reason to treat this as a zero-sum game in which each attempt to do good in the world must crowd out another. For one thing, we’re nowhere near putting all available resources into our efforts to do good, so we can simply choose to expand that budget.

  I am reminded of Zvi’s Slack post (ctrl+f “afford”). Attention is a very scarce resource. If I am spending all my attention on important things, I cannot also spend attention on creating a whole new diet, finding friends who won’t mock me, learning how to cook all new things, etc. On the other hand, animal welfare offsets are probably quite cheap.

  For another, our psychology is complicated, and making a moral effort can just as easily increase our capacity to make further such efforts as deplete it.

  Indeed, and this is actually why I have become mostly vegan in recent months; but it is not going to be true for everyone. My current decision to eat mostly vegan except when inconvenient feels somehow indulgent.

  I wrote more about how I am trying to be vegan: http://​​www.lincolnquirk.com/​​2022/​​02/​​15/​​vegan.html

• lincolnquirkJan 30, 2022, 4:50 PM

  −10 points

  on: Where can I read about the Roko’s Basilisk drama?

  This community has a virtue of taking weird ideas seriously. Roko came up with a weird idea which, the more seriously you took it, the more horrifying it became. This was deemed an info hazard, and censored in some way, I don’t know how. But the people who didn’t take it seriously in the first place weren’t horrified by the idea and thus were confused about why it should have been censored, and thus boosted the Streisand effect.

• lincolnquirkJan 29, 2022, 6:09 PM

  2 points

  on: Mildly Pho­tochromic Lenses?

  I had photochromics for several years. I found them mildly-helpful-and-mostly-unobjectionable in the summer, but ridiculously annoying in the winter (when they both tend to be darker because of low-altitude sun, and the temperature makes them clear up slower once you move inside).

  Also, I was relentlessly mocked by the fashion police. :P

  Ultimately I moved away from them.

• lincolnquirkJan 21, 2022, 9:06 PM

  14 points

  on: Con­sume fic­tion wisely

  I downvoted this. I usually like the concise writing style exhibited in this essay (similar to lsusr, paul graham, both of whom I like) , but I apparently only like it when I think it’s correct. :P

  I especially downvoted because I think it is fairly likely to attract low-quality discussion. A differently-written version of a similar but perhaps more nuanced point, with better fleshed-out examples of why given works are net helpful or net harmful, would be a better post. I am sympathetic to the general idea of the post!

• lincolnquirkDec 27, 2021, 6:16 PM

  1 point

  on: Why did com­puter sci­ence get so galaxy-brained?

  I think there’s something about programming that attracts the right sort of people. What could that be? Well, programming has very tight feedback loops, which make it fun. You can “do a lot”: one’s ability to gain power over the universe, if you will, is quite high with programming. I’d guess a combination of these two factors.