Joachim Bartosik

• Joachim BartosikApr 1, 2023, 5:37 AM

  3 points

  2

  on: Wizards and prophets of AI [draft for com­ment]

  Remember that no matter what, we’re all going to die eventually, until and unless we cure aging itself.

  Not necessarily, there are other options. For example cryonics.

  Which I think is important. If our only groups of options were:

  1) Release AGI which risks killing all humans with high probability or

  2) Don’t do until we’re confident it’s pretty safe it and each human dies before they turn 200.

  I can see how some people might think that option 2) guarantees universe looses all value for them personally and choose 1) even if it’s very risky.

  However we have also have the following option:

  3) Don’t release AGI until we’re confident it’s pretty safe. But do our best to preserve everyone so that they can be revived when we do.

  I think this makes waiting much more palatable—even those who care only about some humans currently alive are better off waiting with releasing AGI it’s at least as likely to succeed as cryonics.

  (also working directly on solving aging while waiting on AGI might have better payoff profile than rushing AGI anyways)

• Joachim BartosikFeb 9, 2023, 10:06 PM

  1 point

  0

  in reply to: Gordon Seidoh Worley’s comment on: Reli­gion is Good, Actually

  For example, you suggest religion involves a set of beliefs matching certain criteria. But some religions really don’t care what you believe! All they ask is that you carry out their rituals. Others ask for faith but not belief, but this is really weird if all you have is a Christian framing where faith is exclusively considered with respect to beliefs.

  Could you give some examples of such religions (that are recognized by many people as religions, not matching definition of religion from the post)?

• Joachim BartosikJan 7, 2023, 12:02 AM

  4 points

  0

  in reply to: Valentine’s comment on: Slack mat­ters more than any outcome

  I don’t feel this way about something like, say, taking oral vitamin D in the winter. That’s not in opposition to some adaptive subsystem in me or in the world. It’s actually me adapting to my constraints.

  If someone’s relationship to caffeine were like that, I wouldn’t say it’s entropy-inducing.

  I think this answers a question /​ request for clarification I had. So now I don’t have to ask.

  (The question was something like “But sometimes I use caffeine because I don’t want to fall asleep while I’m driving (and things outside my controll made it so that doing a few hundred of driving km now-ish is the best option I can see)”).

• Joachim BartosikDec 6, 2022, 10:05 PM

  3 points

  0

  in reply to: paulfchristiano’s comment on: Ver­ifi­ca­tion Is Not Easier Than Gen­er­a­tion In General

  But in that case we just apply verification vs generation again. It’s extremely hard to tell if code has a security problem, but in practice it’s quite easy to verify a correct claim that code has a security problem. And that’s what’s relevant to AI delegation, since in fact we will be using AI systems to help oversee in this way.

  I know you said that you’re not going to respond but in case you feel like giving a clarification I’d like to point out that I’m confused here.

  Yes it usually easy to verify that a specific problem exists if the exact problem is pointed out to you[1].

  But it’s much harder to verify claim that there are no problems, this code is doing exactly what you want.

  And AFAIK staying in a loop:

  1) AI tells us “here’s a specific problem”

  2) We fix the problem then

  3) Go back to step 1)

  Doesn’t help with anything? We want to be in a state where AI says “This is doing exactly what you want” and we have reasons to trust that (and that is hard to verify).

  EDIT to add: I think I didn’t make it clear enough what clarification I’m asking for.

  • Do you think it’s possible to use AI which will point out problems (but which we can’t trust when it says everything is ok) to “win”? It would be very interesting if you did and I’d love to learn more.

  • Do you think that we could trust AI when it says that everything is ok? Again that’d be very interesting.

  • Did I miss something? I’m curious to learn what but that’s just me being wrong (but that’s not new path to win interesting).

  ---

  Also it’s possible that there are two problems, each problem is easy to fix on its own but it’s really hard to fix them both at the same time (simple example: it’s trivial to have 0 false positives or 0 false negatives when testing for a disease; it’s much harder to eliminate both at the same time).

  [1] Well it can be hard to reliably reproduce problem, even if you know exactly what the problem is (I know because I couldn’t write e2e tests to verify some bug fixes).

• Joachim BartosikDec 6, 2022, 12:04 PM

  17 points

  7

  in reply to: Lao Mein’s comment on: Ver­ifi­ca­tion Is Not Easier Than Gen­er­a­tion In General

  What examples of practical engineering problems actually have a solution that is harder to verify than to generate?

  My intuition says that we’re mostly engineering to avoid problems like that, because we can’t solve them by engineering. Or use something other than engineering to ensure that problem is solved properly.

  For example most websites don’t allow users to enter plain html. Because while it’s possible to write non-harmful html it’s rather hard to verify that a given piece of html is indeed harmless. Instead sites allow something like markdown or visual editors which make it much easier to ensure that user-generated content is harmless. (that’s example of engineering to avoid having to verify something that’s very hard to verify)

  Another example is that some people in fact can write html for those websites. In many places there is some process to try and verify they’re not doing anything harmful. But those largely depend on non-engineering to work (you’ll be fired and maybe sued if you do something harmful) and the parts that are engineering (like code reviews) can be fooled because they rely on assumption of your good intent to work (I think; I’ve never tried to put harmful code in any codebase I’ve worked with; I’ve read about people doing that).

• Joachim BartosikAug 14, 2022, 5:49 PM

  5 points

  0

  on: Against Rely­ing on Evolu­tion to Fore­cast AI Out­comes (Part 1)

  I’m confused. What is the outer optimization target for human learning?

  My two top guesses below.

  To me it looks like human values are result of humans learning from environment (which was influenced by humans before and includes current humans). So it’s kind of like human values are what humans learned by definition. So observing that humans learned human values doesn’t tell us anything.

  Or maybe you mean something like parents /​ society /​ … teaching new humans their values? I see some other problems there:

  • I’m not sure what’s success rate but values seem to be changing noticeably

  • There was a lot of time to test multiple methods of teaching new humans values, with humans not changing that much.

• Joachim BartosikMay 3, 2022, 2:20 PM

  1 point

  in reply to: Lukas_Gloor’s comment on: Why hu­mans don’t learn to not rec­og­nize dan­ger?

  This doesn’t always work: sometimes people develop an avoidance to going to the doctor or thinking about their health problems because of this sort of wireheading.

  Yes, but I’d like to understand how sometimes it does work.

• Joachim BartosikMay 3, 2022, 1:06 PM

  2 points

  on: Why hu­mans don’t learn to not rec­og­nize dan­ger?

  I think I was thinking about this post. I’m still interested in learning where I could learn more about this (I now can try to backtrack from the post but since it links to a debate it might be hard to get to sources).

[Question] Why hu­mans don’t learn to not rec­og­nize dan­ger?

• Joachim BartosikApr 23, 2022, 9:49 PM

  2 points

  in reply to: Mahdi Complex’s comment on: On Suc­cess­ful Com­mu­ni­ca­tion Across a Wide In­fer­en­tial Distance

  Yes, I felt that I was missing a point, thank you for pointing to the thing you found interesting in it.

  it’s easier to put yourself into the other person’s ontology and get the message across in terms that they would understand, rather than trying to explain all of science.

  Is a thing that makes sense. But I think the quote doesn’t point at it very well. First a big chunk of it is asserting that belief in witchcraft theory of disease is similar to belief in germ theory of disease. (I don’t know how well average person understands what are viruses)

  Second where it talks about convincing by making concepts similar it’s weird. For example

  … people give untreated well water to their babies. The children regularly get diarrhea, and many of them die (...) if they boil the water, it will kill these bacteria. A month later she’s back, and they’re still giving the babies the dirty water. After all, if a stranger came into your community and told you that your children got influenza because of witchcraft, would you respond by going out and slaughtering a sheep?

  Influenza is much smaller risk than cholera (quick search says CFR for untreated cholera 25-50%, for flu 0.1%) and boiling water is much less costly than slaughtering sheep (it’s likely to result in prison time where I live). (EDIT to add: I didn’t check those numbers so don’t trust them too much, they’re just first numbers I could find and they roughly match my expectations)

  Again thanks for explaining. (at least for me) your comment made the point much better than the quote in the post.

• Joachim BartosikApr 22, 2022, 9:54 PM

  1 point

  on: On Suc­cess­ful Com­mu­ni­ca­tion Across a Wide In­fer­en­tial Distance

  I’m not sure what you find interesting about the quote but I think it’s pretty badly mistaken in trying to make it look like belief in witchcraft is very similar to belief in viruses.

  When people get sick for unaccountable reasons in Manhattan, there is much talk of viruses and bacteria. Since doctors do not claim to be able to do much about most viruses, they do not put much effort into identifying them. Nor will the course of a viral infection be much changed by a visit to the doctor. In short, most appeals in everyday life to viruses are like most everyday appeals to witchcraft. They are supported only by a general conviction that sickness can be explained, and the conviction that viruses can make you sick.

  Two things look wrong to me here:

  • It’s pretty important to distinguish bacterial infections (for which there can be effective treatment) from viral infections (where I think often there is no treatment). (I’m pretty sure witchcraft theory of sickness can’t tell the difference very well) (also vaccines are a things and work against some viruses, AFAIK witchcraft doesn’t have anything similar)

  • If you suspect you have a viral infection and you care to avoid infecting others you can take an effective preventive measures. Not sure if witchcraft theory of sickness could help you.

  If you ask most people in Manhattan why they believe in viruses, they will say two kinds of things: First, they will appeal to authority. “Science has shown,” they will say, though if you ask them how science showed it, you will pretty quickly reach an impasse

  I’m not in Manhattan so I’m not sure if is counts. (I’m also not a virologist) But it should be relatively easy to design experiments to demonstrate that some diseases are transmitted by replicating things. First check if you can infect one subject with (a thing like a bit of snot) from a sick subject. Then try infect a bunch of subjects with diluted (thing like a bit of snot) to check how much you can lower the dose before it stops being infectious. Then take (a thing like a bit of snot) that received ~minumum infective dose, dilute it a bit (so if it was a non-replicating thing like poison dose would be too low to cause symptoms), infect some more subjects & you’re done.

  (actually running this would be probably pretty hard)

  This gets you only that a replicating thing is causing the infection. To check that it’s a virus… well you need a really good microscope so you can identify all the really small things, something to let you separate them and check which kind of the small thing is causing infection.

  (also as far as I can tell this is asking much less from witchcraft; I’m not sure if witchcraft can tell difference between poisoning, bacterial infection, parasite infection, viral infection, vitamin deficiency, ….) (some of those are much easier to distinguish and treat than many viral infections)

• Joachim BartosikApr 10, 2022, 3:28 PM

  7 points

  0

  on: Time-Time Tradeoffs

  I’ve seen the idea in this post:

  Every now and then, you’ll have an opportunity to get great leverage on your money, your time, your energy, your friends, your internet connection, and so forth. Most days, the value of free time is relatively low, and can even be negative (“I’m bored!”), but when you need that time, you really need it. Money isn’t that important most of the time, but when you need it and don’t have it, it’s really bad. Most of the time being low energy, or not having as many friends as you’d like, or having a spotty internet connection, is mostly harmless, but at the wrong time it can spell disaster.

  Using time now to save time later is often efficient even if you spend more time than you save, because that time later on might be orders of magnitude more important.

  I think it’s very important idea and one many people seem to not understand. For example back people complain that mandatory software updates disrupt their work. But usually you have some day to install any update, you could trigger update when you finish your work on first day it’s available and never be forced to update when you don’t want to.

• Joachim BartosikDec 22, 2021, 1:57 PM

  1 point

  on: Zvi’s Thoughts on the Sur­vival and Flour­ish­ing Fund (SFF)

  you presumably don’t hand out any company credit cards at least outside of special circumstances.

  This reminded me of an anecdote from “Surely You’re Joking, Mr. Feynman!” where Feynman says that he

  had been used to giving lectures for some company or university or for ordinary people, not for the government. I was used to, “What were your expenses?” “Soandso much.” “Here you are, Mr. Feynman.

  I remember reading that and thinking that it’s different from what I have to do (at a private company) when I want to expense something. I wonder if things were really done differently back then. And how people made it work.

• Joachim BartosikNov 22, 2021, 9:03 AM

  2 points

  in reply to: Dave Orr’s comment on: The Meta-Puzzle

  “V jbefuvc fngna naq V’z zneevrq.” ?

• Joachim BartosikNov 20, 2021, 1:02 PM

  1 point

  on: Which booster shot to get and when?

  One more thing you might want to consider are vaccine certificates.

  Where I live certificates are valid for a year and booster shots renews a certificate. Also where I live one becomes eligible for a booster shot 6 months after final vaccine dose. So if one gets booster shot ASAP then one gets 18 months of valid certificate. If one delays booster shot until the last moment then one gets 24 month of a valid certificate.

  And valid certificate is very useful over here so there is a real trade off between making one safer against infection vs making more actions available in the future.

  I think it kind of sucks that this is a tradeoff one has to consider.

• Joachim BartosikNov 16, 2021, 3:41 PM

  1 point

  in reply to: Zvi’s comment on: At­tempted Gears Anal­y­sis of AGI In­ter­ven­tion Dis­cus­sion With Eliezer

  I have only a very vague idea what are different reasoning ways (vaguely related to “fast and effortless “ vs “slow and effortful in humans? I don’t know how that translates into what’s actually going on (rather than how it feels to me)).

  Thank you for pointing me to a thing I’d like to understand better.

• Joachim BartosikNov 15, 2021, 5:56 PM

  4 points

  in reply to: lalaithion’s comment on: At­tempted Gears Anal­y­sis of AGI In­ter­ven­tion Dis­cus­sion With Eliezer

  I was thinking that current methods could produce AGI (because Turing-complete) and they can apparently good at producing some algorithms so they might be reasonably good at producing AGI.

  2nd part of that wasn’t explicit for me before your answer so thank you :)

• Joachim BartosikNov 15, 2021, 5:27 PM

  4 points

  on: At­tempted Gears Anal­y­sis of AGI In­ter­ven­tion Dis­cus­sion With Eliezer

  >Which is basically this: I notice my inside view, while not confident in this, continues to not expect current methods to be sufficient for AGI, and expects the final form to be more different than I understand Eliezer/​MIRI to think it is going to be, and that theAGI problem (not counting alignment, where I think we largely agree on difficulty) is ‘harder’ than Eliezer/​MIRI think it is.

  Could you share why you think that current methods are not sufficient to produce AGI?

  Some context:

  After reading Discussion with Eliezer Yudkowsky on AGI interventions I thought about the question “Are current methods sufficient to produce AI?” for a while. I thought I’d check if neural nets are Turing-complete and quick search says they are. To me this looks like a strong clue that we should be able to produce AGI with current methods.

  But I remembered reading some people who generally seemed better informed than me having doubts.

  I’d like to understand what those doubts are (and why there is apparent disagreement on the subject).

• Joachim BartosikOct 26, 2021, 11:12 AM

  10 points

  on: No, re­ally, can “dead” time be sal­vaged?

  First I’ll echo what many others said. You need to rest so be careful to not make things worse (by not resting properly and as a result performing worse at work /​ school /​ whatever you do in your “productive time”).

  That said. If you feel like you’re wasting time then it’s ok to improve that. Some time ago I felt like I was wasting a big chunk of my time. What worked for me was trying out a bunch of things.

  Doing chores. Cooking, cleaning my apartment, replacing my clothes with new ones, maintaining my car. Learning how to get better at chores, in a low effort way. I watched a bunch of youtube videos about how to clean better, how to do laundry better, a ton of recipes. I tried some of those (maybe 1% which looked like it’s the least effort /​ most fun). I like having comfortable clothes, clean apartment, working car. I like some food I can cook better than anything I can buy. So sometimes when I feel tired I enjoy doing chores (since I’m doing them for myself, nobody is forcing me to do them, I can stop doing them whenever I feel like it they are slightly pleasant (very different from when I was doing them on somebody else schedule)).

  Reading blogs, watching educational videos. I count things like “videos about game exploits” [1] cooking videos [2], urban planning related videos[3] as educational videos. I count reading blog posts about history or analysing logistics of Lord of the Rings as good things to read[4].

  Light exercise. I like walking and going on walks helps me a lot with staying healthy.

  Things you’ll enjoy while you’re resting are probably different than those I enjoy so I’d just try a bunch of things which sound like you might like them and see what sticks.

  [1] They’re fun examples of things working as implemented and very much not working as intended.

  [2] I never cook most of them but they’re often fun to watch and sometimes I find something I want to try.

  [3] Also fun to watch and I think they help me understand better why I like some places and make it easier to pick a nice place to live.

  [4] Because they’re taking ideas seriously and it’s helps me with learning to notice when things don’t make sense.

• Joachim BartosikOct 26, 2021, 10:47 AM

  3 points

  in reply to: adamShimi’s comment on: No, re­ally, can “dead” time be sal­vaged?

  Actually, this is heavily criticized by almost anyone sensible in the field: see for example this post by Nate Soares, director of MIRI.

  The link is broken. Did you mean to link to this post?