Thank you for your comment, Zac. The links you suggest will be helpful for me to check whether this kind of analysis has been tried. Up to now I’ve only seen studies directed to interpret specific neurons or areas of a model, but not a statistical analysis of the whole model that can raise an alert when the model is using certain areas previously associated with negative behaviors.
Escaque 66
Karma: −14
- Escaque 66 12 Jul 2023 11:21 UTC1 point0in reply to: Zac Hatfield-Dodds’s comment on: fMRI LIKE APPROACH TO AI ALIGNMENT / DECEPTIVE BEHAVIOUR
For a work implementin this idea, see: https://www.anthropic.com/index/decomposing-language-models-into-understandable-components