Jacob, or “Jisk” when there are too many Jacobs about and I need a nickname.
Honestly pretty disappointed with the state of the modern LW site, but it’s marginally better than other non-blogs so I’m still here.
It should be possible to easily find me from the username I use here, though not vice versa, for interview reasons.
https://philpapers.org/rec/ARVIAA
This paper uses famous problems from philosophy of science and philosophical psychology—underdetermination of theory by evidence, Nelson Goodman’s new riddle of induction, theory-ladenness of observation, and “Kripkenstein’s” rule-following paradox—to show that it is empirically impossible to reliably interpret which functions a large language model (LLM) AI has learned, and thus, that reliably aligning LLM behavior with human values is provably impossible.
So, this seems provisionally to be bullshit because it doesn’t admit of thinking probabilistically or simplicity priors. But I’m not totally sure it’r worthless. Anyone read it in detail?
The older deck sucks. It contains the entirety of the essay without regard to what’s important. This deck is still messy—including too much focusing on the ordering and numbering of the virtues—but it’s significantly superior, and contains concise hearts of the matter. If you’re trying to create a memory aid for the Twelve Virtues, this deck was absolutely an improvement.
If there are a lot of people for the very-low-context NY meetup, possibly at least one very-low-context meetup per quarter is worth doing, to see if that gets people in/back more?
Like others, apparently “think like a mathematician” is enough to get it to work.
Not only is there not a standard name for this set of numbers, but it’s not clear what that set of numbers is. I consulted a better mathematician in the past, and he said that if you allow multiplication it becomes an known unsolved problem whether its representations are unique and whether it can construct all algebraic numbers.
If you give it the up-front caveat “this can represent all rational numbers and at least some algebraic irrationals”, I think that rules out the polynomial appromixation approach, since you can’t give arbitrary arguments and get intermediate values by continuity. But I’m not certain of that.
Yep, that works for Gemini 2.5 as well, got it in one try. In fact, just “think like a mathematician” is enough. Post canceled, everybody go home.
Yes, figure out the notation. The test I gave the LLMs to be sure their solutions weren’t secretly the same as mine in different language was to ask them to properly encode 30,000 and (210)^(2/5).
I meant to leave this link in that footnote. It’s really quite awful.
Sure, done.
Can anyone provide an example conversation (or prefix thereof) which leads to a ‘Nova’ state? I’m finding it moderately tricky to imagine, not being the kind of person who goes looking for it.
Paraphrasing Eddington: If your theory of morality is incompatible with factory farming, then so much the worse for factory farming. If it says not to touch the trolley problem, well, even nominally-obvious thought experiments can be wrong sometimes. But if it says to run a risk of death for all humanity for animals or minds that don’t share human values, there is no hope for it; so much the worse for the theory, at best, or so much for morality at worst.
I find them visually awful and disable them in settings. And avoid using archive.is because there’s no way to turn that off.
Not that I browse LW that much, in fairness.
Doesn’t this imply that having a theory of the domain you’re experimenting in is of low to no value? I find that hard to believe, and therefore doubt your assumptions are correct and applicable.