I like this. It’s like a structural version of control evaluations. Will think where to put it in
technicalities
Karma: 988
One big omission is Bengio’s new stuff, but the talk wasn’t very precise. Sounds like Russell:
With a causal and Bayesian model-based agent interpreting human expressions of rewards reflecting latent human preferences, as the amount of compute to approximate the exact Bayesian decisions increases, we increase the probability of safe decisions.
Another angle I couldn’t fit in is him wanting to make microscope AI, to decrease our incentive to build agents.
I care a lot! Will probably make a section for this in the main post under “Getting the model to learn what we want”, thanks for the correction.
Thanks!
Appendices to the live agendas
Shallow review of live agendas in alignment & safety
I’m not seeing anything here about the costs of data collection (for licenced stuff) or curation (probably hundreds of thousands of cheap hours?), apart from one bullet on OAI’s combined costs. As a total outsider I would guess this could move your estimates by 20-100%.
ActAdd: Steering Language Models without Optimization
ICF is the only such mental viz whizz technique that has ever worked for me, and I say that having done CFAR, a dedicated focussing retreat, a weekend vipassana retreat, and a dedicated circling retreat.
From context I think he meant not fibre laser but “free-space optics”, a then-hyped application of lasers to replace radio. I get this from him mentioning it in the same sentence as satellites and then comparing lasers to radio: “A continuing advance of communications satellites, and the use of laser beams for communication in place of electric currents and radio waves. A laser beam of visible light is made up of waves that are millions of times shorter than those of radio waves”. So I don’t think this rises above the background radiation (ha) of Asimov’s vagueness.
As for 3D TV, if I expand the context you see it’s an explicit replacement for screens: “wall screens will have replaced the ordinary set; but transparent cubes will be making their appearance in which three-dimensional viewing will be possible. In fact, one popular exhibit at the 2014 World’s Fair will be such a 3-D TV, built life-size, in which ballet performances will be seen. The cube will slowly revolve for viewing from all angles.” Also my understanding is that our 3D TVs don’t allow any varying POV, let alone all angles.
Thanks! Added these to the changelog.
Good reason to apply this with nearly equal intensity to mainstream medical arguments, though. (Applies to a lesser extent to evidence-based places like Cochrane, but sadly still applies.)
Good catch! The book is generally written as the history of the world leading up to 2000, and most of its predictions are about that year. But this is clearly an exception and the section offers nothing more precise than “By the year 3000, then, it may well be that Earth will be only a small part of the human realm.” I’ve moved it to the “nonresolved” tab.
DM me for your bounty ($10)! I added your comment to the changelog. Thanks!
Data collector here. Strongly agree with your general point: most of these entries are extremely far from modern “clairvoyant” (cleanly resolving) forecasting questions.
Space travel. Disagree. In context he means mass space travel. The relevant lead-up is this:
“According to her, the Moon is a great place and she wants us to come visit her.”
“Not likely!” his wife answers. “Imagine being shut up in an air—conditioned cave.”
“When you are Aunt Jane’s age, my honey lamb, and as frail as she is, with a bad heart thrown in, you’ll go to the Moon and like it.”
Re: footnote 1. He was a dishonest bugger in his old age so I don’t doubt he would argue that.
Central piloting. Yep, you’re right. We caught this before, but changed it in the wrong branch of the data. Going to make it ‘ambiguous’; let me know if that seems wrong.
Commercial interplanetary travel. Disagree—“C.O.D.” is an old-timey word meaning something so normal and cheap that you don’t even need to pay for your ticket upfront—which implies that “you” is a consumer, not a government. (But again I see what you’re saying.)
DM me for your bounty ($10)! I’ve linked to your comment in the changelog. Thanks!
Is the point that 1) AGI specifically is too weird for normal forecasting to work, or 2) that you don’t trust judgmental forecasting in general, or 3) that respectability bias swamps the gains from aggregating a heavily selected crowd, spending more time, and debiasing in other ways?
The OpenPhil longtermists’ respectability bias seems fairly small to me; their weirder stuff is comparable to Asimov (but not Clarke, who wrote a whole book about cryptids).
And against this, you have to factor in the Big Three’s huge bias towards being entertaining instead of accurate (as well as e.g. Heinlein’s inability to admit error).
Can you point at examples? (Bio anchors?)
Ta!
I’ve added a line about the ecosystems. Nothing else in the umbrella strikes me as direct work (Public AI is cool but not alignment research afaict). (I liked your active inference paper btw, see ACS.)
A quick look suggests that the stable equilibrium things aren’t in scope—not because they’re outgroup but because this post is already unmanageable without handling policy, governance, political economy and ideology. The accusation of site bias against social context or mechanism was perfectly true last year, but no longer, and my personal scoping should not be taken as indifference.
Of the NSF people, only Sharon Li strikes me as doing things relevant to AGI.
Happy to be corrected if you know better!