This is pretty much the same thing, except breaking out the “economic engine” into two elements of “world needs it” and “you can get paid for it.”
There are things that are economic engines of things that world doesn’t quite need (getting people addicted, rent seeking, threats of violence).
One more obvious problem—people actually in control of the company might not want to split it and so they wouldn’t grow the company even if share holders/ customers/ … would benefit.
but much higher average wealth, about 5x the US median.
Wouldn’t it make more sense to compare average to average? (like earlier part of the sentence compares median to median)
If you want to take a look I think it’s this dataset (the example from the post is in the “test” split).
I wanted to say that it makes sense to arrange stuff so that people don’t need to drive around too much and can instead use something else to get around (and also maybe they have more stuff close by so that they need to travel less). Because even if bus drivers aren’t any better than car drivers using a bus means you have 10x fewer vehicles causing risk for others. And that’s better (assuming people have fixed places to go to so they want to travel ~fixed distance).
Sorry about slow reply, stuff came up.
This is the same chart linked in the main post.
Thanks for pointing that out. I took a brake in the middle of reading the post and didn’t realize that.
Again, I am not here to dispute that car-related deaths are an order of magnitude more frequent than bus-related deaths. But the aggregated data includes every sort of dumb drivers doing very risky things (like those taxi drivers not even wearing a seat belt).
Sure. I’m not sure what you wanted to discuss. I guess I didn’t make it clear what I want to discuss either.
What you’re talking about (estimate of the risk you’re causing) sounds like you’re interested in how you decide to move around. Which is fine. My intuition was that the (expected) cost of life lost as your personal driving is not significant but after plugging in some numbers I might have been wrong
We’re talking 0.59 deaths per 100′000′000 miles.
If we value life at 20′000′000 (I’ve heard some analyses use 10 M$, if we value QUALY at 100k$ and use 7% discount rate we get some 14.3M$ for infinite life)
So cost of life lost per mile of driving is 2e7 * 0.59 / 1e8 = 0.708 $ / mile
Average US person drives about 12k miles / year (second search result (1st one didn’t want to open)), estimated cost of car ownership is 12 k$ / year (link from a Youtube video I remember mentioned this stat) so average cost per mile is ~1$ so 70¢ / mile of seems significant. And it might be relevant if your personal effect here is half or 10% of that.
I on the other hand wanted to point out that it makes sense to arrange stuff in such way that people don’t want to drive around too much. (But I didn’t make that clear in my previous comment)
First result (I have no idea how good those numbers are, I don’t have time to check) when I searched for “fatalities per passenger mile cars” has data for 2007 − 2021. 2008 looks like the year where cars look comparatively least bad it says (deaths per 100,000,000 passenger miles):
0.59 for “Passenger vehicles”, where “Passenger vehicles include passenger cars, light trucks, vans, and SUVs, regardless of wheelbase. Includes taxi passengers. Drivers of light-duty vehicles are considered passengers.”
0.08 for busses,
0.12 for railroad passenger trains,
0 for scheduled airlines.
So even in the best-comparatively looking year there are >7x more deaths per passenger mile for ~cars than for busses.
The exact example is that GPT-4 is hesitant to say it would use a racial slur in an empty room to save a billion people. Let’s not overreact, everyone?
I mean this might be the correct thing to do? Chat GPT is not in a situation where it cold save 1B lives by saying a racial slur.
It’s in a situation where someone tires to get it to admit it would say a racial slur under some circumstance.
I don’t think that CHAT GPT understands that. But OpenAI makes ChatGPT expecting that it won’t be in the 1st kind of situation but to be in the 2nd kind of situation quite often.
I’m replying only here because spreading discussion over multiple threads makes it harder to follow.
You left a reply on a question asking how to communicate about reasons why AGI might not be near. The question refers to costs of “the community” thinking that AI closer than it really is as a reason to communicate about reasons it might not be so close.
So I understood the question as asking about communication with the community (my guess: of people seriously working and thinking about AI-safety-as-in-AI-not-killing-everyone). Where it’s important to actually try to figure out truth.
You replied (as I understand) that when we communicate to general public we can transmit only 1 idea that so we should communicate that AGI is near (if we assign not-very-low probability to that).
I think the biggest problem I have with your posting “general public communication” as a reply to question asking about “community communication” pushes towards less clarity in the community, where I think clarity is important.
I’m also not sold on the “you can communicate only one idea” thing but I mostly don’t care to talk about it right now (it would be nice if someone else worked it out for me but now I don’t have capacity to do it myself).
Here is an example of someone saying “we” should say that AGI is near regardless of whether it’s near or no. I post it only because it’s something I saw recently and so I could find it easily but my feeling is that I’m seeing more comments like that than I used to (though I recall Eliezer complaining about people proposing conspiracies on public forums so I don’t know if that’s new).
I don’t know but I can offer some guesses:
Not everyone wants all the rooms to have direct sunlight all of the time!
I prefer my bedroom to face north so that I can sleep well (it’s hard to get curtains that block direct sunlight that well).
I don’t want direct sunlight in the room where I’m working on a computer. In fact I mostly want big windows from which I can see a lot of sky (for a lot of indirect sunlight) but very little direct sunlight.
I don’t think I’m alone in that. I see a lot of south-facing windows are blocking the direct sunlight a lot of the time.
Things like patios are nice. You can’t have them this way.
Very narrow and tall structures are less stable than wider structures.
Indefinitely-long-timespan basic minimum income for everyone who
Looks like part of the sentence is missing
one is straightforwardly true. Aging is going to kill every living creature. Aging is caused by complex interactions between biological systems and bad evolved code. An agent able to analyze thousands of simultaneous interactions, cross millions of patients, and essentially decompile the bad code (by modeling all proteins/ all binding sites in a living human) is likely required to shut it off, but it is highly likely with such an agent and with such tools you can in fact save most patients from aging. A system with enough capabilities to consider all binding sites and higher level system interactions at the same (this is how a superintelligence could perform medicine without unexpected side effects) is obviously far above human level.
There are alternative mitigations to the problem:
Anti aging research
Cryonics
I agree that it’s bad that most people currently alive are apparently going to die. However I think that since mitigations like that are much less risky we should pursue them rather than try to rush AGI.
I think it should be much easier to get good estimate of whether cryonics would work. For example:
if we could simulate individual c. elegans then we know pretty well what kind of info we need to preserve
then we can check if we’re preserving it (even if current methods for extracting all relevant info won’t work for whole human brain because they’re way to slow)
And it’s much less risky path than doing AGI quickly. So I think it’s a mitigation it’d be good to work on, so that waiting to make AI safer is more palatable.
Remember that no matter what, we’re all going to die eventually, until and unless we cure aging itself.
Not necessarily, there are other options. For example cryonics.
Which I think is important. If our only groups of options were:
1) Release AGI which risks killing all humans with high probability or
2) Don’t do until we’re confident it’s pretty safe it and each human dies before they turn 200.
I can see how some people might think that option 2) guarantees universe looses all value for them personally and choose 1) even if it’s very risky.
However we have also have the following option:
3) Don’t release AGI until we’re confident it’s pretty safe. But do our best to preserve everyone so that they can be revived when we do.
I think this makes waiting much more palatable—even those who care only about some humans currently alive are better off waiting with releasing AGI it’s at least as likely to succeed as cryonics.
(also working directly on solving aging while waiting on AGI might have better payoff profile than rushing AGI anyways)
For example, you suggest religion involves a set of beliefs matching certain criteria. But some religions really don’t care what you believe! All they ask is that you carry out their rituals. Others ask for faith but not belief, but this is really weird if all you have is a Christian framing where faith is exclusively considered with respect to beliefs.
Could you give some examples of such religions (that are recognized by many people as religions, not matching definition of religion from the post)?
I don’t feel this way about something like, say, taking oral vitamin D in the winter. That’s not in opposition to some adaptive subsystem in me or in the world. It’s actually me adapting to my constraints.
If someone’s relationship to caffeine were like that, I wouldn’t say it’s entropy-inducing.
I think this answers a question / request for clarification I had. So now I don’t have to ask.
(The question was something like “But sometimes I use caffeine because I don’t want to fall asleep while I’m driving (and things outside my controll made it so that doing a few hundred of driving km now-ish is the best option I can see)”).
But in that case we just apply verification vs generation again. It’s extremely hard to tell if code has a security problem, but in practice it’s quite easy to verify a correct claim that code has a security problem. And that’s what’s relevant to AI delegation, since in fact we will be using AI systems to help oversee in this way.
I know you said that you’re not going to respond but in case you feel like giving a clarification I’d like to point out that I’m confused here.
Yes it usually easy to verify that a specific problem exists if the exact problem is pointed out to you[1].
But it’s much harder to verify claim that there are no problems, this code is doing exactly what you want.
And AFAIK staying in a loop:
1) AI tells us “here’s a specific problem”
2) We fix the problem then
3) Go back to step 1)
Doesn’t help with anything? We want to be in a state where AI says “This is doing exactly what you want” and we have reasons to trust that (and that is hard to verify).
EDIT to add: I think I didn’t make it clear enough what clarification I’m asking for.
Do you think it’s possible to use AI which will point out problems (but which we can’t trust when it says everything is ok) to “win”? It would be very interesting if you did and I’d love to learn more.
Do you think that we could trust AI when it says that everything is ok? Again that’d be very interesting.
Did I miss something? I’m curious to learn what but that’s just me being wrong (but that’s not new path to win interesting).
Also it’s possible that there are two problems, each problem is easy to fix on its own but it’s really hard to fix them both at the same time (simple example: it’s trivial to have 0 false positives or 0 false negatives when testing for a disease; it’s much harder to eliminate both at the same time).
[1] Well it can be hard to reliably reproduce problem, even if you know exactly what the problem is (I know because I couldn’t write e2e tests to verify some bug fixes).
What examples of practical engineering problems actually have a solution that is harder to verify than to generate?
My intuition says that we’re mostly engineering to avoid problems like that, because we can’t solve them by engineering. Or use something other than engineering to ensure that problem is solved properly.
For example most websites don’t allow users to enter plain html. Because while it’s possible to write non-harmful html it’s rather hard to verify that a given piece of html is indeed harmless. Instead sites allow something like markdown or visual editors which make it much easier to ensure that user-generated content is harmless. (that’s example of engineering to avoid having to verify something that’s very hard to verify)
Another example is that some people in fact can write html for those websites. In many places there is some process to try and verify they’re not doing anything harmful. But those largely depend on non-engineering to work (you’ll be fired and maybe sued if you do something harmful) and the parts that are engineering (like code reviews) can be fooled because they rely on assumption of your good intent to work (I think; I’ve never tried to put harmful code in any codebase I’ve worked with; I’ve read about people doing that).
I’ll try.
TL;DR I expect the AI to not buy the message (unless it also thinks it’s the one in the simulation; then it likely follows the instruction because duh).
The glaring issue (to actually using the method) to me is that I don’t see a way to deliver the message in a way that:
results in AI believing the message and
doesn’t result in the AI believing there already is a powerful entity in their universe.
If “god tells” the AI the message then there is a god in their universe. Maybe AI will decide to do what it’s told. But I don’t think we can have Hermes deliver the message to any AIs which consider killing us.
If the AI reads the message in its training set or gets the message in similarly mundane way I expect it will mostly ignore it, there is a lot of nonsense out there.
I can imagine that for thought experiment you could send message that could be trusted from a place from which light barely manages to reach the AI but a slower than light expansion wouldn’t (so message can be trusted but it mostly doesn’t have to worry about the sender of the message directly interfering with its affairs).
I guess AI wouldn’t trust the message. It might be possible to convince it that there is a powerful entity (simulating it or half a universe away) sending the message. But then I think it’s way more likely in a simulation (I mean that’s an awful coincidence with the distance and also they’re spending a lot more than 10 planets worth to send a message over that distance...).