samshap

• samshapJun 3, 2023, 3:27 AM

  1 point

  0

  in reply to: Aaron_Scher’s comment on: Shut­down-Seek­ing AI

  I don’t see how this is more of a risk for a shutdown-seeking goal, than it is for any other utility function that depends on human behavior.

  If anything, the right move here is for humans to commit to immediately complying with plausible threats from the shutdown-seeking AI (by shutting it down). Sure, this destroys the immediate utility of the AI, but on the other hand it drives a very beneficial higher level dynamic, pushing towards better and better alignment over time.

NTIA—AI Ac­countabil­ity Announcement

• samshapFeb 12, 2023, 3:06 PM

  3 points

  4

  in reply to: tgb’s comment on: Top YouTube chan­nel Ver­i­ta­sium re­leases video on Sleep­ing Beauty Problem

  That assumption literally changes the nature of the problem, because the offer to bet, is information that you are using to update your posterior probability.

  You can repair that problem by always offering the bet and ignoring one of the bets on tails. But of course that feels like cheating—I think most people would agree that if the odds makers are consistently ignoring bets on one side, then the odds no longer reflect the underlying probability.

  Maybe there’s another formulation that gives 1:1 odds, but I can’t think of it.

• samshapNov 23, 2022, 7:51 PM

  1 point

  0

  in reply to: paulfchristiano’s comment on: AI will change the world, but won’t take it over by play­ing “3-di­men­sional chess”.

  To the second point, because humans are already general intelligences.

  But more seriously, I think the monolithic AI approach will ultimately be uncompetitive with modular AI for real life applications. Modular AI dramatically reduces the search space. And I would contend that prediction over complex real life systems over long-term timescales will always be data-starved. Therefore being able to reduce your search space will be a critical competitive advantage, and worth the hit from having suboptimal interfaces.

  Why is this relevant for alignment? Because you can train and evaluate the AI modules independently, individually they are much less intelligent and less likely to be deceptive, you can monitor their communications, etc.

• samshapNov 23, 2022, 3:58 AM

  3 points

  −3

  in reply to: paulfchristiano’s comment on: AI will change the world, but won’t take it over by play­ing “3-di­men­sional chess”.

  I take issue with the initial supposition:

  • How could the AI gain practical understanding of long-term planning if it’s only trained on short time scales?

  • Writing code, how servers work, and how users behave seen like very different types of knowledge, operating with very different feedback mechanisms and learning rules. Why would you use a single, monolithic ‘AI’ to do all three?

• samshapNov 22, 2022, 1:35 AM

  1 point

  0

  in reply to: Adam Jermyn’s comment on: Eng­ineer­ing Monose­man­tic­ity in Toy Models

  My weak prediction is that adding low levels of noise would change the polysemantic activations, but not the monosemantic ones.

  Adding L1 to the loss allows the network to converge on solutions that are more monosemantic than otherwise, at the cost of some estimation error. Basically, the network is less likely to lean on polysemantic neurons to make up small errors. I think your best bet is to apply the L1 loss on the hidden layer and the output later activations.

• samshapNov 20, 2022, 2:28 AM

  1 point

  0

  on: By De­fault, GPTs Think In Plain Sight

  I’ve been thinking along very similar lines, and would probably generalize even further:

  Hypothesis: All DNNs thus far developed are basically limited to system-1 like reasoning.

• samshapNov 19, 2022, 10:55 PM

  1 point

  0

  on: Eng­ineer­ing Monose­man­tic­ity in Toy Models

  Great stuff!

  Do you have results with noisy inputs?

  The negative bias lines up well with previous sparse coding implementations: https://​​scholar.google.com/​​citations?view_op=view_citation&hl=en&user=JHuo2D0AAAAJ&citation_for_view=JHuo2D0AAAAJ:u-x6o8ySG0sC

  Note that in that research, the negative bias has a couple of meanings/​implications:

  • It should correspond to the noise level in your input channel.

  • Higher negative biases directly contribute to the sparsity/​monosemanticty of the network.

  Along those lines, you might be able to further improve monosemanticity by using the lasso loss function.

• samshapOct 28, 2022, 11:44 PM

  1 point

  0

  in reply to: RHollerith’s comment on: The harms you don’t see

  Yes, but that was decades ago, when Yeltsin was president! The ‘union state’ has been moribund since the early aughts.

• samshapOct 25, 2022, 3:25 AM

  4 points

  0

  on: AI re­searchers an­nounce Neu­roAI agenda

  I have some technical background in neuromorphic AI.

  There are certainly things that the current deep learning paradigm is bad at which are critical to animal intelligence: e.g. power efficiency, highly recurrent networks, and complex internal dynamics.

  It’s unclear to me whether any of these are necessary for AGI. Something, something executive function and global workspace theory?

  I once would have said that feedback circuits used in the sensory cortex for predictive coding were a vital component, but apparently transformers can do similar tasks using purely feedforward methods.

  My guess is that the scale and technology lead of DL is sufficient that it will hit AGI first, even if a more neuro way might be orders of magnitude more computationally efficient.

  Where neuro AI is most useful in the near future is for embodied sensing and control, especially with limited compute or power. However, those constraints would seem to drastically curtail the potential for AGI.

• samshapJun 8, 2022, 2:59 AM

  7 points

  2

  on: AGI Safety FAQ /​ all-dumb-ques­tions-al­lowed thread

  If the world’s governments decided tomorrow that RL was top-secret military technology (similar to nuclear weapons tech, for example), how much time would that buy us, if any? (Feel free to pick a different gateway technology for AGI, RL just seems like the most salient descriptor).

• samshapMay 2, 2022, 2:35 PM

  6 points

  in reply to: sludgepuddle’s comment on: Quick Thoughts on A.I. Governance

  In my model, Chevron and the US military are probably open to AI governance, because: 1 - they are institutions traditionally enmeshed in larger cooperative/​rule-of-law systems, AND 2 - their leadership is unlikely to believe they can do AI ‘better’ than the larger AI community.

  My worry is instead about criminal organizations and ‘anti-social’ states (e.g. North korea) because of #1, and big tech because of #2.

  Because of location, EA can (and should) make decent connective with US big tech. I think the bigger challenge will be tech companies in other countries , especially China.

Com­men­su­rable Scien­tific Paradigms; or, com­putable induction

• samshapApr 8, 2022, 3:42 PM

  1 point

  in reply to: Rafael Harth’s comment on: Good Heart Week Is Over! (What Next?)

  I published an article on induction https://​​www.lesswrong.com/​​posts/​​7x4eGxXL5DMwRwzDQ/​​commensurable-scientific-paradigms-or-computable-induction of decent length/​​complexity that send to have gotten no visibility at all, which I found very discouraging for my desire to ever do so again. I could only find it by checking my user profile!

• samshapMar 8, 2022, 1:21 PM

  9 points

  on: Amer­ica’s In­visi­ble Grave­yard: Un­der­stand­ing the Mo­ral Im­pli­ca­tions of Western sanctions

  I’m downvoting this, not because it’s wrong or because of weak epistemics, but because politics is the mind killer, and this article is deliberately structured to make that worse.

  I believe politically sensitive topics like this can be addressed on less wrong, but the inflammatory headline and first sentence here are just clickbait.

• samshapJan 21, 2022, 2:36 PM

  2 points

  in reply to: Samuel Hapák’s comment on: Too right to write

  Articles are hard! I was lucky enough to be raised bilingual, so I’m somewhat adept at navigating between different article schemes). I won’t claim these are hard and fast rules in English, but:

  1 - ‘Curiosity’ is an abstract noun (e.g. liberty, anger, parsimony). These generally don’t have articles, unless you need some reason to distinguish between subcategories (e.g. ‘the liberty of the yard’ vs. ‘the liberty of the French’)

  2 - ‘Context’ can refer to either a specific context (e.g. ‘see in the proper context’), in which case the articles are included, or the broad category (e.g. ‘context is everything’). ‘see in the context’ is not ungrammatical, but its usually awkward, because without an adjective its unclear which context you are talking about. (And if you were referring to one that was previously established, you would use ‘that context’ or ‘this context’). However, in the particular case of the button, ‘see in the context’ would be acceptable, because the identity of ‘the context’ is clear! I doubt a native English speaker would say that, though, because its not idiomatic.

  3 - ‘hide the previous comment’ is actually correct here! However, in human-machine interfaces, articles, prepositions, and pronouns are often omitted to save space/​mental effort.

• samshapNov 7, 2021, 2:01 AM

  9 points

  on: Speak­ing of Stag Hunts

  I’m confused.

  In the counterfactual where lesswrong had the epistemic and moderation standards you desire, what would have been the result of the three posts in question, say three days after they were first posted? Can you explain why, using the standards you elucidated here?

  (If you’ve answered this elsewhere, I apologize).

  Full disclosure: I read all three of those posts, and downvoted the third post (and only that one), influenced in part by some of the comments to that post.

• samshapNov 3, 2021, 8:00 PM

  5 points

  on: Paths For­ward: Scal­ing the Shar­ing of In­for­ma­tion and Solutions

  “However there’s definitely an additional problem, which is that the fees are going to the city.”

  Money which the city could presumably use to purchase scarce and vital longshoreman labor.

  The city is getting a windfall because it owns a scarce resource. Would you consider this a problem if the port were privately owned?

  What Ryan is calling punishment is just an ECON 101 cost increase.

• samshapOct 22, 2021, 1:32 AM

  13 points

  in reply to: Lethriloth’s comment on: Petrov Day Ret­ro­spec­tive: 2021

  I’m actually ok with the social pressures inherent in the activity. It’s a subtle reminder of the real influence of this community. The fact that this community would enforce a certain norm makes me more likely to be a conscientious objector in contexts with the opposite norm. (This is true of historical C.O.s, who often come from religious communities).

• samshapOct 14, 2021, 8:32 PM

  8 points

  on: Is nuk­ing women and chil­dren cheaper than fire­bomb­ing them?

  I’d highly recommend ‘The Bomber Mafia’ by Malcolm Gladwell on this subject, which details the internal debates of the US Army Air Corps generals during WWII.

  One of the key questions was whether to use the bombers to target strategic industries, or just for general attrition (i.e. firebombing of civilians). Obviously the first one would have been preferable from a humanitarian perspective (and likely would have ended the European War sooner), but it was very difficult to execute in practice.