quila

Karma: 575

i’m looking for others to (textually) discuss alignment ideas with. if potentially interested, please message me on lesswrong.

my ea forum account

my pgp public key, mainly to prevent future LLM impersonation, you can always ask me to sign a dated message. [the private key is currently stored in plaintext within an encrypted drive, so is vulnerable to being read by local programs]:

I have signed no contracts or agreements whose existence I cannot mention.

quila 18 Nov 2024 21:41 UTC
1 point
0
on: Reducing x-risk might be actively harmful
(was this written by chatgpt?)
another crucial consideration here is that a benevolent ASI could do acausal trade to reduce suffering in the unreachable universe.^[1] (comparing the EV of that probability and of the probability of human-caused long-term-suffering is complex / involves speculation about the many variables going into each side)
1. ^
  there’s writing about this somewhere, i’m here just telling you that the possibility / topic exists
  i wrote this about it but i don’t think it’s comprehensive enough https://quila.ink/posts/ev-of-alignment-for-negative-utilitarians/

quila 17 Nov 2024 16:57 UTC
3 points
0
in reply to: notfnofn’s comment on: notfnofn’s Shortform
from ycombinator comments on a post of that:^[1]
(links to comment by ~~original user~~ a different account^[2] continuing the chat):
1. ^
  (sharing just because it’s relevant info)
2. ^
  source: https://old.reddit.com/r/singularity/comments/1gqss21/gemini_freaks_out_after_the_user_keeps_asking_to/lx0v9se/

quila 13 Nov 2024 7:41 UTC
1 point
0
in reply to: RHollerith’s comment on: quila’s Shortform
recommend against supplementing melatonin
why?
i searched Andrew Huberman melatonin and found this, though it looks like it may be an AI generated summary.

quila 13 Nov 2024 3:31 UTC
14 points
−1
on: quila’s Shortform
i might try sleeping for a long time (16-24 hours?) by taking sublingual^[1] melatonin right when i start to be awake, and falling asleep soon after. my guess: it might increase my cognitive quality on the next wake up, like this:
(or do useful computation during sleep, leading to apparently having insights on the next wakeup? long elaboration below)
i wonder if it’s even possible, or if i’d have trouble falling asleep again despite the melatonin.
i don’t see much risk to it, since my day/night cycle is already uncalibrated^[2], and melatonin is naturally used for this narrow purpose in the body.
‘cognitive quality’ is really vague. here’s what i’m really imagining
my unscientific impression of sleep, from subjective experience (though i only experience the result) and speculation i’ve read, is that it does these things:
- integrates into memory what happened in the previous wake period, and maybe to a lesser extent further previous ones
- more separate to the previous wake period, acts on my intuitions or beliefs about things to ‘reconcile’ or ‘compute implicated intuitions’. for example if i was trying to reconcile two ideas, or solve some confusing logical problem, maybe the next day i would find it easier because more background computation has been done about it?
  - maybe the same kind of background cognition that happens during the day, that leads to people having ideas random-feelingly enter their awareness?
  - this is the one i feel like i have some sub-linguistic understanding of how it works in me, and it seems like the more important of the two for abstract problem solving, which memories don’t really matter to. for this reason, a higher proportion of sleep or near-sleep in general may be useful for problem solving.
but maybe these are not done almost as much as they could be, because of competing selection pressures for different things, of which sleep-time computations are just some. (being awake is useful to gather food and survive)
anyways, i imagine that after those happening for a longer time, the waking mental state could be very ‘fresh’ / aka more unburdened by previous thoughts/experiences (bulletpoint 1), and prone to creativity/‘apparently’ having new insights (bulletpoint 2). (there is something it feels like to be in such a state for me, and it happens more just after waking)
1. ^
  takes effect sooner
2. ^
  i have the non-24 hour sleep/wake cycle that harry has in HPMOR. for anyone who also does, some resources:
  from authors note chapter 98:
  Last but not least:
  You know Harry’s non-24 sleep disorder? I have that. Normally my days are around 24 hours and 30 minutes long.
  Around a year ago, some friends of mine cofounded MetaMed, intended to provide high-grade analysis of the medical literature for people with solution-resistant medical problems. (I.e. their people know Bayesian statistics and don’t automatically believe every paper that claims to be ‘statistically significant’ – in a world where only 20-30% of studies replicate, they not only search the literature, but try to figure out what’s actually true.) MetaMed offered to demonstrate by tackling the problem of my ever-advancing sleep cycle.
  Here’s some of the things I’ve previously tried:
  Taking low-dose melatonin 1-2 hours before bedtime
  Using timed-release melatonin
  Installing red lights (blue light tells your brain not to start making melatonin)
  Using blue-blocking sunglasses after sunset
  Wearing earplugs
  Using a sleep mask
  Watching the sunrise
  Watching the sunset
  Blocking out all light from the windows in my bedroom using aluminum foil, then lining the door-edges with foam to prevent light from slipping in the cracks, so I wouldn’t have to use a sleep mask
  Spending a total of ~$2200 on three different mattresses (I cannot afford the high-end stuff, so I tried several mid-end ones)
  Trying 4 different pillows, including memory foam, and finally settling on a folded picnic blanket stuffed into a pillowcase (everything else was too thick)
  Putting 2 humidifiers in my room, a warm humidifier and a cold humidifier, in case dryness was causing my nose to stuff up and thereby diminish sleep quality
  Buying an auto-adjusting CPAP machine for $650 off Craigslist in case I had sleep apnea. ($650 is half the price of the sleep study required to determine if you need a CPAP machine.)
  Taking modafinil and R-modafinil.
  Buying a gradual-light-intensity-increasing, sun alarm clock for ~$150
  Not all of this was futile – I kept the darkened room, the humidifiers, the red lights, the earplugs, and one of the mattresses; and continued taking the low-dose and time-release melatonin. But that didn’t prevent my sleep cycle from advancing 3 hours per week (until my bedtime was after sunrise, whereupon I would lose several days to staying awake until sunset, after which my sleep cycle began slowly advancing again).
  MetaMed produced a long summary of extant research on non-24 sleep disorder, which I skimmed, and concluded by saying that – based on how the nadir of body temperature varies for people with non-24 sleep disorder and what this implied about my circadian rhythm – their best suggestion, although it had little or no clinical backing, was that I should take my low-dose melatonin 5-7 hours before bedtime, instead of 1-2 hours, a recommendation which I’d never heard anywhere before.
  And it worked.
  I can’t *#&$ing believe that #*$%ing worked.
  (EDIT in response to reader questions: “Low-dose” melatonin is 200microgram (mcg) = 0.2 mg. Currently I’m taking 0.2mg 5.5hr in advance, and taking 1mg timed-release just before closing my eyes to sleep. However, I worked up to that over time – I started out just taking 0.3mg total, and I would recommend to anyone else that they start at 0.2mg.)
  other resources: https://slatestarcodex.com/2018/07/10/melatonin-much-more-than-you-wanted-to-know/, https://www.reddit.com/r/N24/comments/fylcmm/useful_links_n24_faq_and_software/

quila 10 Nov 2024 3:49 UTC
3 points
2
in reply to: lesswronguser123’s comment on: quila’s Shortform
if i left out the word ‘trying’ to (not) use it in that way instead, nothing about me would change, but there would be more comments saying that success is not certain.
i also disagree with the linked post^[1], which says that ‘i will do x’ means one will set up a plan to achieve the highest probability of x they can manage. i think it instead usually means one believes they will do x with sufficiently high probability to not mention the chance of failure.^[2] the post acknowledges the first half of this -- «Well, colloquially, “I’m going to flip the switch” and “I’m going to try to flip the switch” mean more or less the same thing, except that the latter expresses the possibility of failure.» -- but fails to integrate that something being said implies belief in its relevance/importance, and so concludes that using the word ‘try’ (or, by extrapolation, expressing the possibility of failure in general) is unnecessary in general.
1. ^
  though its psychological point seems true:
  But if all you want is to “maximize the probability of success using available resources”, then that’s the easiest thing in the world to convince yourself you’ve done.
2. ^
  this is why this wording is not used when the probability of success is sufficiently far (in percentage points, not logits) from guaranteed.

quila 8 Nov 2024 12:50 UTC
33 points
6
on: quila’s Shortform
nothing short of death can stop me from trying to do good.
the world could destroy or corrupt EA, but i’d remain an altruist.
it could imprison me, but i’d stay focused on alignment, as long as i could communicate to at least one on the outside.
even if it tried to kill me, i’d continue in the paths through time where i survived.

quila 8 Nov 2024 5:49 UTC
4 points
0
on: quila’s Shortform
What is malevolence? On the nature, measurement, and distribution of dark traits was posted two weeks ago (and i recommend it). there was a questionnaire discussed in that post which tries to measure the levels of ‘dark traits’ in the respondent.
i’m curious about the results^[1] of rationalists^[2] on that questionnaire, if anyone wants to volunteer theirs. there are short and long versions (16 and 70 questions).
1. ^
  (or responses to the questions themselves)
2. ^
  i also posted the same shortform to the EA forum, asking about EAs

quila 7 Nov 2024 0:30 UTC
2 points
0
on: quila’s Shortform
something i’d be interested in reading: writings about the authors alignment ontologies over time, i.e. from when they first heard of AI till now

quila 4 Nov 2024 6:40 UTC
5 points
4
in reply to: Charlie Steiner’s comment on: Goal: Understand Intelligence
Understanding [how to design] rather than ‘growing’ search/agency-structure would actually equal solving inner alignment, if said structure does not depend on what target^[1] it is intended to be given, i.e. is targetable (inner-alignable) rather than target-specific.^[2]
Such an understanding would simultaneously qualify as of ‘how to code a capable AI’, but would be fundamentally different from what labs are doing in an alignment-relevant way. In this framing, labs are selecting for target-specific structures (that we don’t understand). (Another difference is that, IIRC, Johannes might intend not to share research on this publicly, but I’m less sure after rereading the quote that gave me that impression^[3]).
1. ^
  includes outer alignment goals
2. ^
  If it’s not clear what I mean, reading this about my background model might help, also feel free to ask me questions
3. ^
  from one of Johannes’ posts:
  I don’t have such a convincing portfolio for doing research yet. And doing this seems to be much harder. Usually, the evaluation of such a portfolio requires technical expertise—e.g. how would you know if a particular math formalism makes sense if you don’t understand the mathematical concepts out of which the formalism is constructed?
  Of course, if you have a flashy demo, it’s a very different situation. Imagine I had a video of an algorithm that learns Minecraft from scratch within a couple of real-time days, and then gets a diamond in less than 1 hour, without using neural networks (or any other black box optimization). It does not require much technical knowledge to see the significance of that.
  But I don’t have that algorithm, and if I had it, I would not want to make that publicly known. And I am unsure what is the cutoff value. When would something be bad to publish? All of this complicates things.
  (After rereading this I’m not actually sure what that means they’d be okay sharing or if they’d intend to share technical writing that’s not a flashy demo)

quila 1 Nov 2024 3:34 UTC
7 points
0
on: JargonBot Beta Test
Let us know what you think!
the grey text feels disruptive to normal reading flow but idk why green link text wouldn’t also be, maybe i’m just not used to it. e.g., in this post’s “Curating technical posts” where ‘Curating’ is grey, my mind sees ”<Curating | distinct term> technical posts” instead of [normal meaning inference not overfocused on individual words]
Is this useful, as a reader?
if the authors make sure they agree with all the definitions they allow into the glossary, yes. author-written definitions would be even more useful because how things are worded can implicitly convey things like, the underlying intuition, ontology, or related views they may be using wording to rule in or out.
Whenever an author with 100+ karma saves a draft of a post, our database queries a language model to:
i would prefer this be optional too, for drafts which are meant to be private (e.g. shared with a few other users, e.g. may contain possible capability-infohazards), where the author doesn’t trust LM companies

quila 30 Oct 2024 16:59 UTC
2 points
0
in reply to: Daphne_W’s comment on: The Summoned Heroine’s Prediction Markets Keep Providing Financial Services To The Demon King!
If you think I missed the point, can you explain in more detail?
Here is my model: Demon king buys shares in “The Demon King will attack the Frozen Fortress”, then sends some small technically-an-attack to the fortress so the market resolves yes, and knowing this will be done is not worth the money lost to the Demon King on the market. No serious-battle plans or military secrets are leaked, and more generally the Demon King would only do this if the information revealed weren’t worth the market cost. (i.e. it’s a central kind of prediction market outcome manipulation, i.e. exploiting how this prediction market assumed a kind of metaphysical gap between predictors and the world / knowledge and action)
Do you disagree with this, or think it’s true but misses the point, in which case what was the point?

quila 27 Oct 2024 19:46 UTC
1 point
−3
in reply to: Gordon Seidoh Worley’s comment on: The hostile telepaths problem
For example, most US school children recite the Pledge of Allegiance every day (or at least they used to). I can remember not fully understanding what the words meant until I was in middle school, but I just went along with it. And wouldn’t you know it, it worked! I do have an allegiance to the United States as a concept.
Can you explain how it caused that, and maybe what it feels like?
(I find it alarming that being forced to recite a pledge as a child can actually have that effect—I knew humans were culturally programmable, but not that {forcing someone to say “I endorse x!” when they don’t know what it means nor want to say it} every day would actually cause them to endorse x later on. Actually, I notice I’m skeptical that that was the real cause in your case; what’s your reason for believing it was the cause?)
(No pressure to answer my questions of course—interpret them as statements of curiosity rather than requests in the human/social sense)

quila 26 Oct 2024 23:33 UTC
3 points
1
in reply to: kqr’s comment on: The Summoned Heroine’s Prediction Markets Keep Providing Financial Services To The Demon King!
it helped them anticipate the Demon King’s next moves – it’s not the market’s fault that they couldn’t convert foresight into operational superiority
The demon king only made those moves to profit from the market, they wouldn’t have been made otherwise

quila 21 Oct 2024 9:18 UTC
8 points
2
on: OpenAI defected, but we can take honest actions
If we stand by while OpenAI violates its charter, it signals that their execs can get away with it. Worse, it signals that we don’t care.
what signals you send to OAI execs seems not relevant.
in the case where they really can’t get away with it, e.g. where the state will really arrest them, then sending them signals / influencing their information state is not what causes that outcome.
if your advocacy causes the world to change such that “they can’t get away with it” becomes true, this also does not route through influencing their information state.
OpenAI is seen as the industry leader, yet projected to lose $5 billion this year
i don’t see why this would lead them to downsize, if “the gap between industry investment in deep learning and actual revenue has ballooned to over $600 billion a year”

quila 21 Oct 2024 8:18 UTC
1 point
0
in reply to: Garrett Baker’s comment on: Alexander Gietelink Oldenziel’s Shortform
how? edit: maybe you meant just the first kind

quila 21 Oct 2024 8:03 UTC
1 point
0
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform
oh i meant medical/covid ones. could also consider furry masks and the cat masks that femboys often wear (e.g. to obscure masculine facial structure), which feel cute rather than ‘cool’, though they are more like the natural human face in that they display an expression (“the face is a mask we wear over our skulls”)

quila 20 Oct 2024 18:27 UTC
12 points
1
in reply to: Alexander Gietelink Oldenziel’s comment on: Alexander Gietelink Oldenziel’s Shortform
also see ashiok from mtg: whole upper face/head is replaced with shadow
also, masks ‘create an asymmetry in the ability to discern emotions’ but do not seem to lead to the rest

quila 18 Oct 2024 4:27 UTC
1 point
0
on: An anti-inductive sequence
What we could do is create a predictor—an algorithm that looks at the previously generated bits, tries to find all possible patterns in them and predict the most likely following bit—and then actually output the opposite. Keep doing this for every bit.
i think a (simplicity-biased) predictor would narrow in on the situation described: that {the rule generating the sequence} contains {a copy of the predictor}, making them irresolvably mutually-dependent, similar to the mutual dependence in the classical halting problem.
in such a case, the predictor is not predicting a 1 or a 0, but a situation where neither can be yielded. so, to be a true implementation of said predictor, it would need to be able to output some third option representing irresolvable situations.
you’d get some string of bits before the predictor considered [irresolvable-mutual-dependance exception] most probable though! what that string is (for some prediction-narrowing procedure) sounds like a fun question

quila 16 Oct 2024 7:01 UTC
4 points
−1
on: quila’s Shortform
one of my basic background assumptions about agency:
there is no ontologically fundamental caring/goal-directedness, there is only the structure of an action being chosen (by some process, for example a search process), then taken.
this makes me conceptualize the ‘ideal agent structure’ as being “search, plus a few extra parts”. in my model of it, optimal search is queried for what action fulfills some criteria (‘maximizes some goal’) given some pointer (~ world model) to a mathematical universe sufficiently similar to the actual universe → search’s output is taken as action, and because of said similarity we see a behavioral agent that looks to us like it values the world it’s in.
i’ve been told that {it’s common to believe that search and goal-directedness are fundamentally intertwined or meshed together or something}, whereas i view goal-directedness as almost not even a real thing, just something we observe behaviorally when search is scaffolded in that way.
if anyone wants to explain the mentioned view to me, or link a text about it, i’d be interested.
(maybe a difference is in the kind of system being imagined: in selected-for systems, i can understand expecting things to be approximately-done at once (i.e. within the same or overlapping strands of computations); i guess i’d especially expect that if there’s a selection incentive for efficiency. i’m imagining neat, ideal (think intentionally designed rather than selected for) systems in this context.)
edit: another implication of this view is that decision theory is its own component (could be complex or not) of said ‘ideal agent structure’, i.e. that superintelligence with an ineffective decision theory is ~~possible~~ (edit: nontrivially likely for a hypothetical AI designer to unintentionally program / need to avoid). that is, one being asked the wrong questions (i.e. of the wrong decision theory) in the above model.
What links here?
- quila's comment on Goal: Understand Intelligence by Johannes C. Mayer (4 Nov 2024 6:40 UTC; 5 points)

quila 7 Oct 2024 9:53 UTC
3 points
0
in reply to: Richard_Kennaway’s comment on: quila’s Shortform
yep not contesting any of that
neither is there in rationality a recipe with which you can just crank the handle and come up with a proof of a conjecture
to be clear, coming up with proofs is a central example of what i meant by creativity. (“they are not satisfied by avoiding failure conditions, but require the satisfaction of some specific, hard-to-find success condition”)