Currently completely random yes. We experimented with a more intelligent “daemon manager,” but it was hard to make one which didn’t have a strong universal preference for some daemons over others (and the hacks we came up with to try to counteract this favoritism became increasingly convoluted). It would be great to find an elegant solution to this.
Good point! Thanks for letting people know.
I’ve also had that problem, and whenever I look through the suggestions I often feel like there were many good questions/comments that got pruned away. The reason to focus on surprise was mainly to avoid the repetitiveness caused by mode collapse, where the daemon gets “stuck” giving the same canned responses. This is a crude instrument though, since as you say, just because a response isn’t surprising, doesn’t mean it isn’t useful.
Simple randomness seems bad because it’ll lead to oversampling some daemons and starving others. Why not simply rotate through a shuffled list? (The user could also drag-and-drop an order of speakers.)
We experimented with a more intelligent “daemon manager”...It would be great to find an elegant solution to this.
Seems like ideally you’d want something like a Mixture of Experts approach—a small, fast model that gets info about which daemons are best at what, along with your most recent input, and picks the right one.
Thanks!
Replying in order:
Currently completely random yes. We experimented with a more intelligent “daemon manager,” but it was hard to make one which didn’t have a strong universal preference for some daemons over others (and the hacks we came up with to try to counteract this favoritism became increasingly convoluted). It would be great to find an elegant solution to this.
Good point! Thanks for letting people know.
I’ve also had that problem, and whenever I look through the suggestions I often feel like there were many good questions/comments that got pruned away. The reason to focus on surprise was mainly to avoid the repetitiveness caused by mode collapse, where the daemon gets “stuck” giving the same canned responses. This is a crude instrument though, since as you say, just because a response isn’t surprising, doesn’t mean it isn’t useful.
Simple randomness seems bad because it’ll lead to oversampling some daemons and starving others. Why not simply rotate through a shuffled list? (The user could also drag-and-drop an order of speakers.)
Seems like ideally you’d want something like a Mixture of Experts approach—a small, fast model that gets info about which daemons are best at what, along with your most recent input, and picks the right one.