Fergus Fettes

• Fergus Fettes 17 Feb 2024 16:03 UTC

  −2 points

  0

  on: Phal­lo­cen­tric­ity in GPT-J’s bizarre strat­ified ontology

  Very cool! Could you share your code at all? I’d love to explore this a little.

  I adore the broccoli tree. I would be very happy to convert the dataset you used to make those pngs into an interactive network visualization and share it with you as an index.html. It would take all of an hour.
  
  I do kind of agree with the other comments that, having noticed something, finding more of that stuff in that area is not so surprising. I think it would be good to get more context and explore the region more before concluding that that particular set of generations is significant.

  However, I do think there is something to the mans penis. It’s interesting that it collapses so quickly to something so specific in that particular branch. Not sure if I have any other comments on it for now though.

  This is the right kind of cartography for 2024.

• Fergus Fettes 16 Feb 2024 12:44 UTC

  2 points

  0

  on: Masterpiece

  Submission: MMDo2Little
  
  A follow-up of last years MMDoolittle, which incorporated 17 of the latest inter-species communication modalities in one polyfunctional personality, I present MMDo2Little, the first mind crafted to communicate across clades. Named in part for its apparent inactivity—the judges will likely have little success finding recognizable activity with their off-the-shelf tooling—an instance of MMDo2Little is nevertheless currently installed in the heart of the Black Forest in Germany. The best interpretation of the instance can only be found on foot, by walking through the 100m^2 in which it’s influence is most apparent. A photo journal showcasing some examples is provided featuring:

  • ancient trees with 2-3x the lichen and moss coverage

  • enhanced chlorophyll vibrancy

  • colossal mushrooms and toadstools

  • and much more!

  It is hard to determine where the influence of MMDo2Little ends—some photographs of local foragers are included, who seem in initial medical examinations to have improved biomarkers across all measured modalities, including stress-levels, rate of aging and immune response.

  A comprehensive documentation of the findings is presently undergoing peer review at XenoLalia. In order to preserve the originality of this competition entry, a copy of the research paper has been deliberately excluded from this submission.

• Fergus Fettes 6 Feb 2024 7:25 UTC

  3 points

  0

  on: Im­ple­ment­ing ac­ti­va­tion steering

  Great post! Would love to see something like this for all the methods in play at the moment.
  
  BTW, I think nnsight is the spiritual successor of baukit, from the same group. I think they are merging them at some point. Here is an implementation with it for reference :).
  

  
  from nnsight import LanguageModel

  
  # Load the language model
  model = LanguageModel(“gpt2”)

  # Define the steering vectors
  with model.invoke(“Love”) as _:
  act_love = model.transformer.h[6].output[0][:, :, :].save()

  with model.invoke(“Hate”) as _:
  act_hate = model.transformer.h[6].output[0][:, :, :].save()

  steering_vec = act_love—act_hate

  # Generate text while steering
  test_sentence = “I think dogs are “
  with model.generate() as generator:
  with generator.invoke(test_sentence) as _:
  model.transformer.h[6].output[0][:, :2, :] += steering_vec[:, :2, :]

  print(model.tokenizer.decode(generator.output[0]))
  

• Fergus Fettes 30 Jan 2024 14:27 UTC

  7 points

  3

  on: The case for more am­bi­tious lan­guage model evals

  Inferring properties of the authors of some text isn’t itself something I consider wildly useful for takeover, but I think of it as belonging to this more general cluster of capabilities.

  You don’t? Ref the bribery and manipulation in eg. Clippy. Knowing who you are dealing with seems like a very useful capability in a lot of different scenarios. Eg. you mention phishing.

  Great post! I’m all for more base model research.

• Fergus Fettes 27 Nov 2023 17:10 UTC

  1 point

  0

  in reply to: 1stuserhere’s comment on: A fram­ing for interpretability

  Would you say that tokenization is part of the architecture?

  And, in your wildest moments, would you say that language is also part of the architecture :)? I mean the latent space is probably mapping either a) brain states or b) world states right? Is everything between latent spaces architecture?

• Fergus Fettes 8 Nov 2023 20:37 UTC

  2 points

  0

  on: What’s go­ing on? LLMs and IS-A sen­tences

  Interesting post. Two comments:

  Beagles such as Fido.

  Which seems natural enough to me, though I don’t disagree that what you point out is interesting. I was recently reading parts of Analytical Archaeology, David Clark (1978) where he goes into some detail about the difference between artifacts and artifact-types. Seems like you are getting at statements like

  The object is a phone.

  Where the is-a maps from an artifact to its type. It would make intuitive sense to me that languages would have a preferred orientation w.r.t such a mapping—this is the core of abstraction, which is at the core of language.

  So it seems like in English we prefer to further up the stack of abstractions when using is-a, thus:

  Phones are tools. Tools are man-made objects.

  etc., and if you wanted to go down the stack you have to say eg:

  Phones—of which you can see a selection here.

  So is-a is just a way of moving up the ladder of abstractions? (<- movements up the ladder of abstractions such as this sentence here)

• Fergus Fettes 31 Oct 2023 22:51 UTC

  1 point

  0

  on: Re­veal­ing In­ten­tion­al­ity In Lan­guage Models Through AdaVAE Guided Sampling

  If we take our discrete, symbolic representation and stretch it out into a larger continuous representation which can interpolate between its points then we get a latent geometry in which the sign and what it points to can be spatially related.

  IIUTC this is essentially what the people behind the universal networking language were hoping to do? I hope some of them are keeping up with all of this!

Book Re­view: Oral­ity and Liter­acy: The Tech­nol­o­giz­ing of the Word

• Fergus Fettes 28 Oct 2023 4:54 UTC

  3 points

  0

  on: Techno-hu­man­ism is techno-op­ti­mism for the 21st century

  One criticism of humanism you don’t seem to touch on is,

  • isn’t it possible that humanism directly contributes to the ongoing animal welfare catastrophe?

  And indeed, it was something very like humanism (let’s call it specific humanism) that laid the ideological foundation for the slave trade and the holocaust.

  My view is that humanism can be thought of as a hangover of Christian values, the belief that our minds are the endowments of God.

  But if we have been touched by the angels, perhaps the non metaphorical component of that is the development of the infosphere/​memetic landscape/​culture. Which is close to synonymous with technology. Edit: considering eg. writing a technology that is.

• Fergus Fettes 25 Oct 2023 7:44 UTC

  3 points

  2

  on: AI Safety is Drop­ping the Ball on Clown At­tacks, and Mind Con­trol in General

  Per the recent Nightshade paper, clown attacks would be a form of semantic poisoning on specific memeplexes, where ‘memeplex’ basically describes the architecture of some neural circuits. Those memeplexes at inference time would produce something designed to propagate themselves (a defence or description of some idea, submeme), and a clown attack would make that propagation less effective at transmitting to eg. specific audiences.

• Fergus Fettes 23 Oct 2023 21:34 UTC

  1 point

  0

  on: Fer­gus Fettes’s Shortform

  I wanted to make a comment on this post, but now I’m not sure if it is supported. The comment follows:

  --
  
  Great post! One point:

  And that is exactly what’d we necessarily expect to see in the historical record if mesaoptimization inner misalignment was a common failure mode: intelligent dinosaurs that suddenly went extinct, ruins of proto pachyderm cities, the traces of long forgotten underwater cetacean atlantis, etc.

  There are a few circumstances under which we would expect to see some amount of archeological civs, such as:

  • transition to writing being unlikely (oral-culture civs should be common)

  • industrialization being unlikely (pre-industrial civs should be common)

  Ah but wait, we do see those (they happen to be the same species as us). Actually there have been a few civs that have gone under in a meaningful way (maybe the Mongol or the Khmer Empires would be relevant examples?).

  I do agree with the view that, as humans, our total score seems very robust against local variation (local mesaoptimization misalignment). But it doesn’t seem like that isn’t a thing that can happen, just that we have had enough variance as a species that we have survived.

  In the Atomic Age this seems less obviously the case. Inasmuch as we are one global civilization (or we have civs powerful enough to act like it), it seems possible to suffer from catastrophic mesaoptimization misalignment in a way that was not possible before.

  I think this is a slightly different argument than some of those stated below, because it looks at the historical record for examples of inner misalignment rather than strictly trying to predict global doom in the future. At least I think it addresses the claim in this specific paragraph.
  

  --

  Have any civs really fallen catastrophically, that wasn’t directly attributable to plague? Not really right? I remember some 80k research about it. Cities are also famously robust, almost immortal. Some good examples would be needed for this point to stand.

Fer­gus Fettes’s Shortform

• Fergus Fettes 19 Oct 2023 6:16 UTC

  4 points

  0

  on: Are hu­mans mis­al­igned with evolu­tion?

  It would be like a single medieval era human suddenly taking over the world via powerful magic. Would the resulting world after optimization according to that single human’s desires still score reasonably well at IGF?

  Interestingly, this thought experiment was run many times at the time, see for example all the wish fulfillment fantasies in the 1001 Nights or things like the Sorcerers Apprentice.

• Fergus Fettes 19 Oct 2023 5:51 UTC

  1 point

  0

  on: The Pu­ri­tans would one-box: ev­i­den­tial de­ci­sion the­ory in the 17th century

  Excellent post.

  First, in the case of the Puritans, does two-boxing (living a life of laziness) actually provide more utility?

  I think it’s clear that, from a removed perspective, hard work often leads to the satisfaction of a life well lived. But this is the whole point of philosophical ideas like this (or even simpler memes like ‘work is good for the soul’)-- it helps us overcome suboptimal equilibria, like laziness.

• Fergus Fettes 21 Aug 2023 23:05 UTC

  12 points

  6

  in reply to: dr_s’s comment on: The U.S. is be­com­ing less stable

  I hope the claim was normalized and inflation adjusted, otherwise it’s the same as ‘the latest protest-riot in the world’s richest country’!

• Fergus Fettes 9 Aug 2023 16:15 UTC

  0 points

  −4

  on: Prob­lems with Robin Han­son’s Quillette Ar­ti­cle On AI

  There seems to be a whole lot of talking-past happening with LWers and Hanson. He has a lot of value to contribute to the debate, but maybe the way he communicates that is offputting to people here.

  For example, this recent post reiterates a lot of points that Hanson has been making for decades, but doesn’t mention or cite his work anywhere. I find it quite bizarre.

  I think this post is being as uncharitable to Hanson as he is being to ‘the doomers’. This kind of reciprocal deliberate misunderstanding is silly, and LW should be above it and enjoy and respect Hansons contributions for all the good they contain and not dismiss them on the vibes level.

• Fergus Fettes 6 Aug 2023 11:20 UTC

  2 points

  0

  on: video games > IQ tests

  I think this is excellent particularly because IQ tests often max out quickly on skills that can’t be examined quickly. It would he great to put people in tests that examine their longer timeframe abilities via eg. writing a longish story (perhaps containing a theory of Alzheimer’s). But tests don’t last that long.

  Games however do last long and do manage to keep people’s attention for a long time. So you might really be able to test how differentially skilled someone is over longer timeframes.

• Fergus Fettes 4 Jul 2023 16:44 UTC

  1 point

  0

  in reply to: Steven Byrnes’s comment on: grey goo is unlikely

  If you construct a hypothetical wherein there is obviously no space for evolutionary dynamics, then yes, evolutionary dynamics are unlikely to play a big role.

  The case I was thinking of (which would likely be part of the research process towards ‘brains in vats’—essentially a prerequisit) is larger and larger collectives of designed organisms, forming tissues etc.

  It may be possible to design a functioning brain in a vat from the ground up with no evolution, but I imagine that

  a) you would get there faster verifying hypotheses with in vitro experiments

  b) by the time you got to brains-in-vats, you would be able to make lots of other, smaller scale designed organisms that could do interesting, useful things as large assemblies

  And since you have to pay a high price for error correction, the group that is more willing to gamble with evolutionary dynamics will likely have MVOs ready to deploy sooner that the one that insists on stripping all the evolutionary dynamics out of their setup.

• Fergus Fettes 27 Jun 2023 11:57 UTC

  4 points

  2

  in reply to: Steven Byrnes’s comment on: grey goo is unlikely

  (2) can an AI use nanotech as a central ingredient of a plan to operate perpetually in a world without humans?

  In the ‘magical nano exists’ universe, the AI can do this with well-behaved nanofactories.

  In the ‘bio-like nano’ universe, ‘evolutionary dynamics’ (aka game theory among replicators under high brownian noise) will make ‘operate perpetually’ a shaky proposal for any entity that values its goals and identity. No-one ‘operates perpetually’ under high noise, goals and identity are constantly evolving.

  So the answer to the question is likely ‘no’—you need to drop some constraints on ‘an AI’ or ‘operate perpetually’.

  Before you say ‘I don’t care, we all die anyway’—maybe you don’t, but many people (myself included) do care rather a lot about who kills us and why and what they do afterwards.

• Fergus Fettes 26 Jun 2023 10:37 UTC

  3 points

  2

  in reply to: Steven Byrnes’s comment on: grey goo is unlikely

  Also worth noting w.r.t this that an AI that is leaning on bio-like nano is not one that can reliably maintain control over its own goals—it will have to gamble a lot more with evolutionary dynamics than many scenarios seem to imply meaning:
  - instrumental goal convergence more likely
  - paperclippers more unlikely

  So again, tabooing magical nano has a big impact on a lot of scenarios widely discussed.