Overall, I thought it was very good. I agree that “super optimizer” is more likely to create the correct impression in the average person than “super intelligence”, and will stop using the latter term.
The bit about the “golem genie” seems forced, though—I’m not sure it actually clarifies things. It seems like such a direct analogy; I’d expect that people that understand “superoptimizer” won’t need the analogy, and those who don’t understand, won’t be helped by it. For the latter group of people, it might help to introduce the golem before talking about superoptimization at all. It’s quite possible I’m wrong
Overall, I thought it was very good. I agree that “super optimizer” is more likely to create the correct impression in the average person than “super intelligence”, and will stop using the latter term.
The bit about the “golem genie” seems forced, though—I’m not sure it actually clarifies things. It seems like such a direct analogy; I’d expect that people that understand “superoptimizer” won’t need the analogy, and those who don’t understand, won’t be helped by it. For the latter group of people, it might help to introduce the golem before talking about superoptimization at all. It’s quite possible I’m wrong