Arguing that the consequentialist approach is better than the deontological approach is different than skipping that step and going straight to refuting your own consequentialist argument for the position others were arguing on deontological grounds. Saying they should do some expected utility calculations is different than saying the expected utility calculations the haven’t done are wrong.
I thought he was ramming home his point that his opponents are secretly deontologists there
I think the point was that his opponents are openly deontologists, making openly deontological arguments for their openly deontological position, and therefor they are rightly confused and not moved by Yvain’s refutation of a shoehorning of their position into a consequentialist argument when they never made any such argument, which Yvain now understands and therefor he doesn’t do that anymore.
This seems to be overloading the term “side effects”. The functional programming concept of side effects (which it says its functions shouldn’t have) is changing the global state of the program that invokes them other than by returning the value. It makes no claims of these other concepts of a program being affected by analyzing the source code of the function independent of invoking it or of the the function running on morally relevant causal structure.
CFAR can achieve its goal of creating effective, rational do-gooders by taking existing do-gooders and making them more effective and rational. This is why they offer scholarships to existing do-gooders. Their goal is not to create effective, rational do-gooders out of blank slates but make valuable marginal increases in this combination of traits, often by making people who already rank highly in these areas even better.
They also use the same workshops to make people in general more effective and rational, which they can charge money for to fund the workshops, and gives them more data to test their training methods on. That they don’t turn people in general into do-gooders does not constitute a failure of the whole mission. These activities support the mission without directly fulfilling it.
Fourth, they become a sort of fraternal organization, i.e. membership does bring benefits mainly from being able to network with other members.
CFAR is creating an alumni network to create benefits on top of increased effectiveness and rationality.
CFAR does offer to refund the workshop fee if after the fact participants evaluate that it wasn’t worth it. They also solicit donations from alumni. So they are kind of telling participants to evaluate the value provided by CFAR and pay what they think is appropriate, while providing an anchor point and default which covers the cost of providing the workshop. That anchor point and default are especially important for the many workshop participants who are not selected for altruism, who probably will learn a lot of competence and epistemic rationality but not much altruism, and whose workshop fees subsidize CFAR’s other activities.
my feeling about CFAR is that they are providing a service to individuals for money and it’s probably not a terrible idea to let the market determine if their services are worth the amount they charge.
I think that CFAR’s workshops are self funding and contribute to paying for organizational overhead. Donated funds allow them to offer scholarships to their workshops to budding Effective Altruists (like college students) and run the SPARC program (targeting mathematically gifted children who may be future AI researchers). So, while CFAR does provide a service to individuals for money, donated money buys more services targeted at making altruistic people more effective and getting qualified people working on important hard problems.
I’m convinced AGI is much more likely to be built by a government or major corporation, which makes me more inclined to think movement-building activities are likely to be valuable, to increase the odds of the people at that government or corporation being conscious of AI safety issues, which MIRI isn’t doing.
MIRI’s AI workshops get outside mathematicians and AI researchers involved in FAI research, which is good for movement building within the population of people likely to be involved in creating an AGI.
Qiaochu’s answer seems off. The argument that the parent AI can already prove what it wants the successor AI to prove and therefore isn’t building a more powerful successor, isn’t very compelling because being able to prove things is a different problem than searching for useful things to prove. It also doesn’t encompass what I understand to be the Lobian obstacle, that being able to prove that if your own mathematical system proves something that thing is true implies that your system is inconsistent.
It sounds to me like somebody is purchasing utilons, using themselves as an example to get other people to also purchase utilons, and incidentally deriving a small amount of well deserved status from the process.
Can you give an example of a property a star might have because having that property made its ancestor stars better at producing descendant stars with that property?
However, it’s actually determined what your decision is—any Laplacian demon could deduce it from looking at your brain. It’s all pretty clear, and quantum events are not enough to derail it (barring very very low measure stochastic events). So from the universe’s perspective, you’re not choosing anything, not shifting measure from anything to anything.
The logical structure of my decision still controls what world gets the measure. From Timeless Control:
Surely, if you can determine the Future just by looking at the Past, there’s no need to look at the Present?
The problem with the right-side graph is twofold: First, it violates the beautiful locality of reality; we’re supposing causal relations that go outside the immediate neighborhoods of space/time/configuration. And second, you can’t compute the Future from the Past, except by also computing something that looks exactly like the Present; which computation just creates another copy of the Block Universe (if that statement even makes any sense), it does not affect any of the causal relations within it.
This is basically the same point as the one I keep making and you keep missing: The universe/Laplacian demon/whatever is adding quantum measure, in order to select the same world to add measure to that was selected by your decision, it has to duplicate the causal structure of your decision and the resulting world. (And since within this computation the same things happen for the same reasons as in the selected world, by the generalized anti zombie principle, the computation is adding measure to that world even at times before your model says it adds quantum measure.)
I don’t think that would prevent you from falling asleep, but you would get more benefit from the nap if it is earlier, so your waking periods are more even. You should also be able to sleep less at night, typical biphasic schedules have a 6.5 hour core.
Are you claiming that a) this model is incoherent, or b) that this model does not entail what I’m claiming (that you should save for the future)?
The basic model you described, even as alternative physics, is underspecified, and depending on how I try to steelman it so it is coherent, it doesn’t entail what you claim, and if I try to steelman it so it entails what you say, it isn’t coherent.
The big question is what worlds get to accumulate measure and why those particular worlds. If the answer is that all worlds accumulate measure, then the accumulation happens independently of your decision, so the effect should not impact your decision. If the answer is that the measure accumulation process looks somehow depends on what world your decision leads to, then the measure accumulation process in locating that world duplicates its causal structure, and by the globalized anti zombie principle, contains all the same conscious people as that world, so it adds to the worlds decision theoretical measure even before your model says it officially adds to its quantum measure (this is basically parallel to the argument for Many Worlds). What I think is incoherent is the idea that you can add measure to world state without adding measure to the process that selected that world state, which you try to do by supposing that your decision (and its intermediate effects) don’t cause the later accumulation of measure, yet the measure magically accumulates in the world that results from your decision. (To account for this, you would have to follow the probability to outside the thought experiment.)
It feels like this should all be obvious if you understand why p-zombies are incoherent, why Many Worlds is obviously correct, and how these are related.
My model was of gradual proportional increase in utility
Yes, my example shows a proportional increase in measure between two times, and is indifferent to the gradual increase between these times. If you think the gradual increase is important, please provide an example that illustrates this.
not absolute addition to every branch.
I have already explained why adding the measure to a single branch is incoherent in both the cases where the decision causes or does not cause selection of the branch that receives the measure.
Yes, that is how your decision gives your measure M to world WA or to world WB, but that shouldn’t affect accumulation of measure into later states of these worlds by quantum fluctuation, so both worlds still get measure 10M from that.
Unless you mean that quantum fluctuations into later states of the world are directed by the normal evolution of the earlier states, including your decision, in which case, this process would be adding measure (perhaps not quantum measure, but counting as decision theoretic measure in the same way) to the initial state of the world in which you make the decision (because it is another instance of the same causal chain, that is, it produces the same result for the same reasons), so you get all 10M of the quantum fluctuation measure right away, and choice A gives 44M utilons while B still gives 33M utilons.
I am not sure what you mean by “logically incompatible worlds”, but if worlds WA and WB are the results of different available decisions of an agent embedded in a common precursor world, then they both follow the same laws of physics and just have their particles or whatever in different places, and in a quantum universe they just have different quantum states.
I was envisaging utilons being “consumed” at the time they were added (say people eating chocolate bars).
My example is entirely compatible with this.
So choosing A would add 4M utilons, and choosing B would add 33M utilons.
So the problem here is that you are not accounting for the fact that choosing A in the measure M world does not prevent the accumulation of measure 10M to world WB from quantum fluctuation. You get those 30M utilons whether you choose A or B, choosing A gets you an immediate 4M additional utilons, while choosing B gets you a deferred 3M utilons.
Arguing that the consequentialist approach is better than the deontological approach is different than skipping that step and going straight to refuting your own consequentialist argument for the position others were arguing on deontological grounds. Saying they should do some expected utility calculations is different than saying the expected utility calculations the haven’t done are wrong.