Thank you for your comment, Zac. The links you suggest will be helpful for me to check whether this kind of analysis has been tried. Up to now I’ve only seen studies directed to interpret specific neurons or areas of a model, but not a statistical analysis of the whole model that can raise an alert when the model is using certain areas previously associated with negative behaviors.
Escaque 66
Karma: −14
For a work implementin this idea, see: https://www.anthropic.com/index/decomposing-language-models-into-understandable-components