Nothing is “mere.” I, too, can see the stars on a desert night, and feel them. But do I see less or more? The vastness of the heavens stretches my imagination—stuck on this carousel, my little eye can catch one-million-year-old light. A vast pattern—of which I am a part—perhaps my stuff was belched from some forgotten star, as one is belching there. Or see them with the greater eye of Palomar, rushing all apart from some common starting point when they were perhaps all together. What is the pattern, or the meaning, or the why? It does not do harm to the mystery to know a little about it.
- Richard P. Feynman on The Relation of Physics to Other Sciences
Dalcy
Karma: 852
Mildly surprised how some verbs/connectives barely play any role in conversations, even in technical ones. I just tried directed babbling with someone, and (I think?) I learned quite a lot about Israel-Pakistan relations with almost no stress coming from eg needing to make my sentences grammatically correct.
Example of (a small part of) my attempt to summarize my understanding of how Jews migrated in/out of Jerusalem over the course of history:
They here *hand gesture on air*, enslaved out, they back, kicked out, and boom, they everywhere.
(audience nods, given common knowledge re: gestures, meaning of “they,” etc)
[Question] Least-problematic Resource for learning RL?
Gearing Up for Long Timelines in a Hard World
Related—“There are always many ways through the garden of forking paths, and something needs only one path to happen.”
Also, davidad’s Open Agency Architecture is a very concrete example of what such a non-antisocial pivotal act that respects the preferences of various human representatives would look like (i.e. a pivotal process).
Perhaps not realistically feasible in its current form, yes, but davidad’s proposal suggests that there might exist such a process, and we just have to keep searching for it.
Agree that current AI paradigm can be used to make significant progress in alignment research if used correctly. I’m thinking something like Cyborgism; leaving most of the “agency” to humans and leveraging prosaic models to boost researcher productivity which, being highly specialized in scope, wouldn’t involve dangerous consequentialist cognition in the trained systems.
However, the problem is that this isn’t what OpenAI is doing—iiuc, they’re planning to build a full-on automated researcher that does alignment research end-to-end, for which orthonormal was pointing out that this is dangerous due to their cognition involving dangerous stuff.
So, leaving aside the problems with other alternatives like pivotal act for now, it doesn’t seem like your points are necessarily inconsistent with orthonormal’s view that OpenAI’s plans (at least in its current form) seem dangerous.
Complaint with Pugh’s real analysis textbook: He doesn’t even define the limit of a function properly?!
It’s implicitly defined together with the definition of continuity where , but in Chapter 3 when defining differentiability he implicitly switches the condition to without even mentioning it (nor the requirement that now needs to be an accumulation point!) While Pugh has its own benefits, coming from Terry Tao’s analysis textbook background, this is absurd!
(though to be fair Terry Tao has the exact same issue in Book 2, where his definition of function continuity via limit in metric space precedes that of defining limit in general … the only redeeming factor is that it’s defined rigorously in Book 1, in the limited context of )
*sigh* I guess we’re still pretty far from reaching the Pareto Frontier of textbook quality, at least in real analysis.
… Speaking of Pareto Frontiers, would anyone say there is such a textbook that is close to that frontier, at least in a different subject? Would love to read one of those.
Any advice on reducing neck and shoulder pain while studying? For me that’s my biggest blocker to being able to focus longer (especially for math, where I have to look down at my notes/book for a long period of time). I’m considering stuff like getting a standing desk or doing regular back/shoulder exercises. Would like to hear what everyone else’s setups are.
Update: huh, nonstandard analysis is really cool. Not only are things much more intuitive (by using infinitesimals from hyperreals instead of using epsilon-delta formulation for everything), by the transfer principle all first order statements are equivalent between standard and nonstandard analysis!
Man, deviation arguments are so cool:
what are macrostates? Variables which are required to make your thermodynamics theory work! If they don’t, add more macrostates!
nonequilibrium? Define it as systems that don’t admit a thermodynamic description!
inductive biases? Define it as the amount of correction needed for a system to obey Bayesian updating, i.e. correction terms in the exponent of the Gibbs measure!
coarse graining? Define the coarse-grained variables to keep the dynamics as close as possible to that of the micro-dynamics!
or in a similar spirit—does your biological system deviate from expected utility theory? Well, there’s discovery (and money) to be made!
It’s easy to get confused and think the circularity is a problem (“how can you define thermodynamics in terms of equilibriums, when equilibriums are defined using thermodynamics?”), but it’s all about carving nature at the right joints—and a sign that you made the right carving is that the amount of corrections needed to be applied aren’t too numerous, and they all seem “natural” (and of course, all of this while letting you make nontrivial predictions. that’s what matters at the end of the day).
Then, it’s often the case that those corrections also turn out to be meaningful and natural quantities of interest.
I used to try out near-random search on ideaspace, where I made a quick app that spat out 3~5 random words from a dictionary of interesting words/concepts that I curated, and I spent 5 minutes every day thinking very hard on whether anything interesting came out of those combinations.
Of course I knew random search on exponential space was futile, but I got a couple cool invention ideas (most of which turned out to already exist), like:
infinite indoor rockclimbing: attach rocks to a vertical treadmill, and now you have an infinite indoor rock climbing wall (which is also safe from falling)! maybe add some fancy mechanism to add variations to the rocks + a VR headgear, I guess.
clever crypto mechanism design (in the spirit of CO2 Coin) to incentivize crowdsourcing of age-reduction molecule design animal trials from the public. (I know what you’re thinking)
You can probably do this smarter now if you wanted, with eg better GPT models.
gwern’s take on a similar paper (Tinystories), in case anyone was wondering. Notable part for me:
...
Now, what would be really interesting is if they could go beyond the in-domain tasks and show something like meta-learning. That’s supposed to be driven by the distribution and variety of Internet-scale datasets, and thus should not be elicited by densely sampling a domain like this.
I wonder if the following is possible to study textbooks more efficiently using LLMs:
Feed the entire textbook to the LLM and produce a list of summaries that increases in granularity and length, covering all the material in the textbook just at a different depth (eg proofs omitted, further elaboration on high-level perspectives, etc)
The student starts from the highest-level summary, and gradually moves to the more granular materials.
When I study textbooks, I spend a significant amount of time improving my mental autocompletion, like being able to familiarize myself with the terminologies, which words or proof-style usually come in which context, etc. Doing this seems to significantly improve my ability to read eg long proofs, since I can ignore all the pesky details (which I can trust my mental autocompletion to later fill in the details if needed) and allocate my effort in getting a high-level view of the proof.
Textbooks don’t really admit this style of learning, because the students don’t have prior knowledge of all the concept-dependencies of a new subject they’re learning, and thus are forced to start at the lowest-level and make their way up to the high-level perspective.
Perhaps LLMs will let us reverse this direction, instead going from the highest to the lowest.
What’s a good technical introduction to Decision Theory and Game Theory for alignment researchers? I’m guessing standard undergrad textbooks don’t include, say, content about logical decision theory. I’ve mostly been reading posts on LW but as with most stuff here they feel more like self-contained blog posts (rather than textbooks that build on top of a common context) so I was wondering if there was anything like a canonical resource providing a unified technical / math-y perspective on the whole subject.
There’s still some pressure, though. If the bites were permanently not itchy, then I may have not noticed that the mosquitos were in my room in the first place, and consequently would less likely pursue them directly. I guess that’s just not enough.
Why haven’t mosquitos evolved to be less itchy? Is there just not enough selection pressure posed by humans yet? (yes probably) Or are they evolving towards that direction? (they of course already evolved towards being less itchy while biting, but not enough to make that lack-of-itch permanent)
this is a request for help i’ve been trying and failing to catch this one for god knows how long plz halptbh would be somewhat content coexisting with them (at the level of houseflies) as long as they evolved the itch and high-pitch noise away, modulo disease risk considerations.
Having lived ~19 years, I can distinctly remember around 5~6 times when I explicitly noticed myself experiencing totally new qualia with my inner monologue going “oh wow! I didn’t know this dimension of qualia was a thing.” examples:
hard-to-explain sense that my mind is expanding horizontally with fractal cube-like structures (think bismuth) forming around it and my subjective experience gliding along its surface which lasted for ~5 minutes after taking zolpidem for the first time to sleep (2 days ago)
getting drunk for the first time (half a year ago)
feeling absolutely euphoric after having a cool math insight (a year ago)
...
Reminds me of myself around a decade ago, completely incapable of understanding why my uncle smoked, being “huh? The smoke isn’t even sweet, why would you want to do that?” Now that I have [addiction-to-X] as a clear dimension of qualia/experience solidified in myself, I can better model their subjective experiences although I’ve never smoked myself. Reminds me of the SSC classic.
Also one observation is that it feels like the rate at which I acquire these is getting faster, probably because of increase in self-awareness + increased option space as I reach adulthood (like being able to drink).
Anyways, I think it’s really cool, and can’t wait for more.
i absolutely hate bureaucracy, dumb forms, stupid websites etc. like, I almost had a literal breakdown trying to install Minecraft recently (and eventually failed). God.
Answering my own question, review / survey articles like https://arxiv.org/abs/1811.12560 seem like a pretty good intro.