Staff Researcher at Epoch. AI forecasting.
Pablo Villalobos
Karma: 519
Compute Trends — Comparison to OpenAI’s AI and Compute
Compute Trends Across Three eras of Machine Learning
Parameter counts in Machine Learning
Let me paraphrase your argument, to see if I’ve understood it correctly:
-
Physical constraints on things such as energy consumption and dissipation imply that current rates of economic growth on Earth are unsustainable in the relatively short term (<1000 years), even taking into account decoupling, etc.
-
There is a strong probability that expanding through space will not be feasible
-
Therefore, we can reasonably expect growth to end some time in the next centuries
First of all, if economic progress keeps being exponential then I think it’s quite possible that technological progress will mostly continue at previous rates.
So in 100-200 years, it seems certainly possible that space expansion will become much easier, if for example genetic engineering allows humans to better tolerate space environments.
But that’s pretty much a “boring world” scenario where things keep going mostly as they are now. I expect the actual state of humanity in 200 years will be extremal: either extinction or something very weird.
Material needs, entertainment, leisure… are basically all covered for most people in rich countries. If you think about what could provide a substantial increase in utility to a very rich person nowadays, I think it’s down to better physical health (up to biological immortality), mental health, protection from risks… and after all of that you pretty much have to start providing enlightenment, eudaimomia or whatever if you want to improve their lives at all.
So when you have a stable population of immortal enlightened billionaires… Well, perhaps you’ve reached the peak of what’s possible and growth is not necessary anymore. Or perhaps you’ve discovered a way to hack physics and energy and entropy are not important anymore.
So, even if 200 years is a short amount of time by historic standards, the next 200 years will probably produce changes big enough that physical constraints that we would reach in 300 years at current trends stop being relevant.
-
So, assuming the neocortex-like subsystem can learn without having a Judge directing it, wouldn’t that be the perfect Tool AI? An intelligent system with no intrinsic motivations or goals?
Well, I guess it’s possible that such a system would end up creating a mesa optimizer at some point.
Survey on cortical uniformity—an expert amplification exercise
“A PP-based AGI would be devilishly difficult to align”
Is this an actual belief or a handwavy plot device? If it’s the first, I’m curious about the arguments
My perspective as a native speaker who doesn’t remember his grammar lessons very well:
The subjunctive mood has a lot of uses, at least in Spain (I’m not really familiar with other varieties of Spanish). Some examples off the top of my head:
1. Counterfactual conditionals: “Si Lee Harvey Oswald no hubiera disparado a JFK, alguien más lo habría hecho” (If Lee Harvey Oswald hadn’t shot JFK, someone else would have), here “no hubiera disparado” is subjunctive and means “hadn’t shot”.
2. To speak about people’s actions or decisions which depend on preferences. “Hará lo que quiera con el dinero” (He’ll do what he wants with the money), here “quiera” is the present subjunctive of “querer”, meaning “to want”.
3. To speak about properties of unknown entities. “Quien pueda trabajar será pagado” (Those who can work will be payed), here “pueda” is the present subjunctive form of “poder”, which means “to be able to”.
Here is a fairly comprehensive list of uses (in Spanish 😉)
I think in general the subjunctive mood conveys some degree of unrealness or subjectivity. You could probably say many of the examples above using indicative mood only, but you would definitely lose some expressive power (I don’t know why this is not the case in other languages)
I remember being super confused when I was learning English because of the lack of a distinct subjunctive verbal form. Say, in “I wish I had had a car back then”, the two “had” have completely different meanings, one for past tense and one for expressing desire. The Spanish equivalent would be “habido” and “hubiera” from the verb “haber” respectively.
Your example is right, but it’s not true that it’s used in all subordinate clauses. For example, “Estoy buscando a la persona que escribió ese libro” (I’m looking for the person who wrote that book) does not have any verb in subjunctive mood.
EA Madrid monthly social
Anders Sandberg: “How to Shape the Future Sensibly”
AI safety reading group in Madrid
Logarithms and Total Utilitarianism
The lecture will take place in classroom B15
Meetup : Madrid—Rationality lecture and activities
Thank you!
Meetup : Madrid—First meetup
I don’t reflect on it. This happens in two ways:
I find reflecting much more cognitively demanding than reading, so if there is a ‘next post’ button or similar, I tend to keep reading.
Also, sometimes when I try to actually think about the subject, it’s difficult to come up with original ideas. I often find myself explaining or convincing an imaginary person, instead of trying to see it with fresh eyes. This is something I noticed after reading the corresponding Sequence.
I guess establishing an habit of commenting would help me solve these problems.
Hello, I’m a math-cs undergrad and aspiring effective altruist, but I haven’t chosen a cause yet. Since that decision is probably one of the most important ones, I should probably wait until I’ve become stronger.
To that end, I’ve read the Sequences (as well as HPMOR), and I would like to attend a CFAR workshop or similar at some point in the future. I think one of my problems is that I don’t actually think that much about what I read. Do you have any advice on that?
Also, there are a couple of LWers in my college with whom I have met twice, and we would like to start organising meetups regularly. Would you please give me some karma so that I can add new meetups? (I promise I will make up for it with good contributions)
Thanks!
I’m sorry, but I find the tone of this post a bit off-putting. Too mysterious for my taste. I opened the substack but it only has one unrelated post.
Do you think that there is a non-secular way forward? Did you previously (before your belief update) think there is a non-secular way forward?
I can agree with this, but… those indicators seem pretty meaningful for me. Life expectancy, poverty rates, etc. And at least now we have indicators! Previously there wasn’t even that!
And why does this kind of misticism attract so much people over here? Why are the standard arguments against religion/magic and for materialism and reductionism not compelling to you anymore?