Roko comments on Truth and Advantage: Response to a draft of “AI safety seems hard to measure”

[Roko]

there are truths you don’t want it noticing or taking advantage of.

Meaning truths about how it could scam the creators in some way? Like, “your alignment system has weakness X which I could exploit”?