I could be way off on this, but I cannot help but core here is less about complexity than it is about efficiency. The most efficient processes do all appear to be a bit simpler than they probably are. It’s a bit like watching an every talented craftsman working and thinking “The looks easy.” Then when you try you find out it was much more difficult and complicated than it appeared. The craftsman’s efficiency in action (ability to handle/deal with the underlying complexity) masked the truth a bit.
jmh
Karma: 1,485
I’ve had similar experiences where my intuition tells me to be cautious but I could not say why. When I’ve ignored those intuitions I’ve generally paid a price. So now I do give them consideration.
In such situations it is probably good to take some time to sit back and try to identify some of the things that triggered the response. We are very good at pattern matching but also really good at filtering. Could be that intuitions like “getting bad vibes” is all about the interaction of the two.
But that is a pretty difficult task, we’re asking our self to go back and review all the details we ignored and filtered. But I suspect it is a very good thing to try doing.
The desires-values schema reminds me of Hirshmann’s The Passions and the Interests and the problem he (IIRC) sees Machiavelli dealing with. Machiavelli is advising kings who he sees as more driven by their passions but to be successful need to be driven more by their interests.
Anyone in the alignment space take a look at Machiavelli in the light of how to get some thing that is more powerful (king versus the advisor) better aligned? Probably need to stretch things a bit to see that type of alignment as being aligned with the general welfare of the kingdom but seems like he was dealing with a similar problem when looked at from certain angles.
I like the point about the need for some type of external competitive measure but as you say, they might not be a MMA gym where you need one.
Shifting the metaphor, I think your observation of the sucker punch fits well with the insight that for those with only a hammer, all problems look like nails. The gym would be someone with a screwdriver or riveter as well as the hammer. But even lacking the external check, we should always ask ourselves “Is this really a nail?” I might only have a hammer but if this isn’t a nail while the results might be better than nothing (maybe?) I’m not likely to achieve the best that could be done some other way. And,of course, watching just what happens went the other hammer-only people solve the problem and compare results to those when we know the problem is a nail we might learn something from their mistakes.
I agree with the view that punishment is not really a great deterrent as many crimes are not committed from a calculated cost-benefit perspective. I do think we need to apply that type of thinking towards what we might do with that insight/fact of things.
On that point, would like to see more on your claim that we would get better bang for the buck as it were from more investment in preventing crimes. In this regard I’m thinking about the contrast between western legal views and places like China as well as the estimates on the marginal pecuniary costs of prevention to the marginal pecuniary savings from reduced punishment. Clearly two (among many) difference margins along which trade-offs will need to be made.
Another aspect that seems worth exploring (and I’m sure it has been but not sure where the literature stands on the question) is how, at least in my understanding of USA criminal law, victims of crimes are not often compensated (white collar, fraud, financial crimes are something of an exception) but the victims, as a part of society, are then paying costs to punish the criminal. Full prevention is not a reasonable assumption (not sure what level is a reasonable assumption) but we might find a better solution even at the current rate of prevention if more of those harmed by crimes were actually compensated for the harms rather than just imposing the punishment of the criminal actor. A primary reason for preventing the event of a crime is the prevention of the harm. But if the harm can be largely mitigated after the fact there is a degree of equivalent between it never having occurred and it’s compensation (perhaps here we might think of law and punishment as a type of insurance).
I also think there is something to look at in terms of prevention of incidence of crimes due to incarceration—a type of exile. There might be scope for approaches there that maintain that type of prevention for repeat offenders (those demonstrating a propensity for some bad behavior for whatever reason) that may be possible at a lower cost than prison incarceration. And what might the marginal gain in prevention be related to the cost of the solution. This is a somewhat different approach than the ex anti prevention approach applied to the general society but may be nearly as effective but substantially lower cost.
Agree. There is that old saying about even fools learning from their own mistakes but wise men learning from the mistakes of others. But if everyone is trying to hide their mistakes, that might limit how much learning the wise can do.
I had not really thought about this before, but after seeing your comment the question struck me if social/cultural norms about social status and “loosing face” don’t impact scientific advancement.
Nice write up and putting some light on something I think I have intuitively been doing but not quite realizing it. Particularly the impact on growth of wealth.
I was thinking that a big challenge for a lot of people is the estimated distribution—which is likely why so many non-technical rationales are given by many people. Trying to assess that is hard and requires a lot of information about a lot of things—something the insurance companies can do (as suggested by another comment) but probably overwhelms most people who buy insurance.
With that thought, I was wondering if anyone has thought of shifting the equations a bit. Rather than working up some estimate of the probability space, why not put an equation together that you might be able to churn out some probability distributions given W, P, d_i and c_i. for the break-even case. I think most people would be able to digest that, event x_i has implied probability p_i, event x_j has implied probability p_j. Then the person can think if those probabilities actually make sense to them and their situation.
Clearly, it could not be an exhaustive listing of events but I would think a table of three or four of the main events that carry the greatest losses would be a good starting point for most people.
What is the price of the past? Kind of leading question but I’ve found myself wondering at times about the old saying about those who don’t know the past are doom to repeat it.
It’s not that I don’t think there is a good point to that view. However, when I look at the world around me I often see something that is vastly different from that view. I’ve come to summarize that as those who cannot let go of the past will never escape it. The implication is that not only those “clingy” people but also those around them will continue living whatever past it is they are attached to. As this is generally bad events of the past that means we get stuck living with these bad outcomes in the world.
So while keeping that knowledge of the past in mind may prevent repeating it, that might be accomplished by never actually having learned from or moved past the history. I’m unsure which is worse.
Yes, all those conjectures are possible as we don’t yet know what the reality will be—it is currently all conjecture.
The counter argument to yours I think is just what opportunities is the AI giving up to do whatever humans might be left to do? What is the marginal value of all the things this ASI might be able to be doing that we cannot yet even conceive of?
I think the suggestion of a negative value is just out of scope here as it doesn’t fit into theory of comparative advantage. That was kind of the point of the OP. It is fine to say comparative advantage will not apply but we lack any proof of that and have plenty of examples where it actually does hold even when there is a clear absolute advantage for one side. Trying to reject the proposition by assuming it away seems a weak argument.
I’m not sure that is the correct take in the context of Comparative Advantage.
It would not matter if the SI could produce more than humans in a direct comparison but what the opportunity cost for the SI might be. If the ASI is shifting efforts that would have produced more value to it than it gets from the $77 sunlight output AND that delta in value is greater than the lower productivity of the humans then the trade makes sense to the ASI.
Seems to me the questions here are about resource constraints and whether or not an ASI does or does not need to confront them in a meaningful way.
You’re touching on one of the questions that occurred to me. What do the current and post-Jones transportation flows look like? While I agree that the law must shift some from shipping to truck, rail or pipeline I’m not sure I would expect massive changes here. Do you have some data on that point?
I think one clear aspect of the stories here, yours and John’s, relates to what I’ll call asymmetric information flows. Basically, the times at which the information, that no one is trying to keep secret, become known to the relevant parties.
Of course understanding what a good update frequency is for various situations should be is a tricky thing itself.
If I’m reading this correctly, then generally we’re seeing a rather flat payoff curve over most “do good opportunities” and the rare max should stand out like a sore thumb when taking a good look. So those really should be things do-gooders will jump on quickly. (Note, that doesn’t mean they are done quickly or that additional assistance is not important.)
While not as obvious, it probably also means that a lot of more mundane opportunities are getting ignored. That comes from an insight offered in one of my classes from years back asking why so much clumping (think fad type stuff here) exists when the marginal utility of the consumed good is pretty much equal to all the other goods that could have been consumer. In other words, when the opportunity cost is zero why is everyone doing the same thing?
I suspect we could see something like that in the “do good” space. Therefore, taking the path not followed could be a very good thing.
One point I’m not sure about with the idea of neutrality is neutrality of process or of outcome. Or would that distinction not matter to your interests here?
Interesting but I’ve just skim so will need to come back. With that caveat made, I seem to have had a couple of thought that keep recurring for me that seem compatible or complementary with your thoughts.
First, where do we define the margin between public and private. It strikes me that a fair amount of social strife does revolve around a tension here. We live in a dynamic world so thinking that the sphere of private actions will remain static seem unlikely but as the world changed (knowledge, applied knowledge driving technology change, movement of people resulting in cultural transmission and tensions...) will be forces resulting in a change in the line between public and private.
While I’m not entirely sure it is the best framing, I do think of this in the form of externalities. Negative externalities are the more challenging form. What I think starts happening is that we live in t=0 and some set of private activities are producing very little negative impacts on others. But we find by t=10 some of the elements in that set of private activities are now producing a large enough total negative external effect that:
People are able to start seeing cause and effect
The costs to others are sufficient to over come the organizational and transactional costs of using social infrastructures to seek relief for the harms. Those can be formal in the sense of courts and government law/regulation requests. But also informal in various forms—the people engaging in the activities loose reputation, don’t get invited to the good parties any more, people don’t want to talk with them or be seen with them any more, perhaps even more aggressive responses.
At some point either most accept that a new definition of “private” exists and the old ways have changed or society reached the point that those who have not adapted will be treated as criminal and removed from society.
The other thing I’ve been thinking about is related. One hears the where’s my flying car, it’s the 21st Century already quip now and then. But I think a better one might be: It’s the 21st Century, why am I still living under and 18th Century form of government?
I think these relate some of your post in that a lot of the social conflict you point to is driven by the shifting margin between public and private sphere of action. As that margin shifts people use the government to address those new conflicts within the society. But few if any governments differ substantially from those that have existed for centuries. I would characterize that vision of government, even when thinking of representative democracies, as that of an actor/agent. Government takes actions, just like the private members of society do. It should function, as you say, in a neutral way. Part of the failing there comes from government, being an actor/agent, then has its own interests, agendas and biases.
That government as an active participant contrasts a bit with how I think most people think of markets. Markets don’t really do anything. They are simply an environment in which active entities come and interact with each other. Markets don’t set price or quality or even really type of item—these are all unplanned outputs. The market itself is indifferent to all those, it’s neutral in the sense you use that term.
Well, in the 21st Century might we not think that how governments are structured might also shift? While I am far from sure that the shift would be correctly called divestiture or privatization (which seems most people think of when talking about fixing government—or for some calling for increasing what its already doing) I do think the shift might be away from an acting entity and more into some type of passive environment that has some commonality with markets. In a very real sense governments are already a type of market setting but not a price/money exchange one (the representatives are not quite but out bids and offers on votes) but clearly these is an demand mediation and supply process going on. But currently the market-like aspect of government is about integrating voter/members of society demands and then the government makes a decision and takes the actions it wants. I would think some areas might be suitable for taking out the government being the actor and let the actions be decentralized among the people. Probably not individual action, I suspect some sub-agent presence will exist to reduce organizational/transaction costs but certainly the process would look more market-like and be a more neural setting. That might well then remove a lot of the divisiveness and conflict we see with the existing “old school” forms of government.
That is all probably a bit poorly written and expressed but it’s a quick dump of a couple of not fully thought out ideas.
Years ago when I was hanging out with day traders there was a heuristic they all seemed to hold. If their trading model was producing winning trades two out of three times they thought the model was good and could be used. No one ever suggested why that particular rate was the shared meme/norm—why not 4 out of 5 or 3 out of 5. I wonder if empirically (or just intuitively over time) they simply approximated the results in this post.
Or maybe just a coincidence, but generally when money is at stake I think the common practices will tend to reflect some fundamental fact of the environment.
Could you clarify a bit here. Is Hanson talking about specific cultures or all of the instances of culture?
Thanks that does help clarify the challenges for me.
I was just scrolling through Metaculus and its predictions for the US Elections. I noticed that pretty much every case was a conditional If Trump wins/If doesn’t win. Had two thought about the estimates for these. All seem to suggest the outcomes are worse under Trump. But that assessment of the outcome being worse is certainly subject to my own biases, values and preferences. (For example, for US voters is it really a bad outcome if the probability of China attacking Taiwan increases under Trump? I think so but other may well see the costs necessary to reduce the likelihood as high for something that is not something that actually involves the USA.)
So my first though was how much bias should I infer as present in these probability estimates? I’m not sure. But that does relate a bit to my other thought.
In one sense you could naively apply the p, therefore not p is the outcome for the other candidate as only two actually exist. But I think it is also clear that the two probability distributions don’t come from the same pool so conceivably you could change the name to Harris and get the exact same estimates.
So I was thinking, what if Metaculus did run the two cases side by side? Would seeing p(Haris) + p(Trump) significantly different than 1 suggest one should have lower confidence in the estimates? I am not sure about that.
What if we see something like p(H) approximately equale to p(T)? does that suggest the selected outcome is poorly chosen as it is largely independant of the elected candidate so the estimates are largely meaninless in terms of election outcomes? I have a stronger sense this is the case.
So my bottome line now is that I should likely not hold a high confidence that the estimates on these outcomes are really meaninful with regards to the election impacts.
With regards to thinking about what comes next, you might find these two links, if you didn’t already come across them, of some interest.
https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/three-worlds-in-2035/ hypothesizes 3 global futures for 2035.
https://www.atlanticcouncil.org/content-series/atlantic-council-strategy-paper-series/welcome-to-2035/ offers results from a survey about various outcomes of states that might obtain in (by???) 2035. I didn’t find much surprising here but some of the questions I had not given thought to before so hardly had a point of view on them.