I think you underestimate simple self-ratings. You could just do those, and yes, there are automated ways. For example, you could turn on a Web browser plugin like RescueTime but disable any blocking functionality—so it’s just tracking time spent. Randomize intervention X for a few months, pull the RescueTime logs, and voila! A (non-blind) randomized experiment.
I think you underestimate simple self-ratings. You could just do those, and yes, there are automated ways. For example, you could turn on a Web browser plugin like RescueTime but disable any blocking functionality—so it’s just tracking time spent. Randomize intervention X for a few months, pull the RescueTime logs, and voila! A (non-blind) randomized experiment.