Shower thought[*]: the notion of a task being bounded doesn’t survive composition. Specifically, say a task is bounded if the agent doing it is only using bounded resources and only optimising a small bit of the world to a limited extent. The task of ‘be a human in the enterprise of doing research’ is bounded, but the enterprise of research in general is not bounded. Similarly, being a human with a job vs the entire human economy. I imagine keeping this in mind would be useful when thinking about CAIS.
Similarly, the notion of a function being interpretable doesn’t survive composition. Linear functions are interpretable (citation: the field of linear algebra), as is the ReLU function, but the consensus is that neural networks are not, or at least not in the same way.
I basically wish that the concepts that I used survived composition.
Fwiw, this seems like an interesting thought but I’m not sure I understand it, and curious if you could say it in different words. (but, also, if the prospect of being asked to do that for your shortform comments feels ughy, no worries)
Often big things are made of smaller things: e.g., the economy is made of humans and machines interacting, and neural networks are made of linear functions and ReLUs composed together. Say that a property P survives composition if knowing that P holds for all the smaller things tells you that P holds for the bigger thing. It’s nice if properties survive composition, because it’s easier to figure out if they hold for small things than to directly tackle the problem of whether they hold for a big thing. Boundedness doesn’t survive composition: people and machines are bounded, but the economy isn’t. Interpretability doesn’t survive composition: linear functions and ReLUs are interpretable, but neural networks aren’t.
Shower thought[*]: the notion of a task being bounded doesn’t survive composition. Specifically, say a task is bounded if the agent doing it is only using bounded resources and only optimising a small bit of the world to a limited extent. The task of ‘be a human in the enterprise of doing research’ is bounded, but the enterprise of research in general is not bounded. Similarly, being a human with a job vs the entire human economy. I imagine keeping this in mind would be useful when thinking about CAIS.
Similarly, the notion of a function being interpretable doesn’t survive composition. Linear functions are interpretable (citation: the field of linear algebra), as is the ReLU function, but the consensus is that neural networks are not, or at least not in the same way.
I basically wish that the concepts that I used survived composition.
[*] Actually I had this on a stroll.
Fwiw, this seems like an interesting thought but I’m not sure I understand it, and curious if you could say it in different words. (but, also, if the prospect of being asked to do that for your shortform comments feels ughy, no worries)
Often big things are made of smaller things: e.g., the economy is made of humans and machines interacting, and neural networks are made of linear functions and ReLUs composed together. Say that a property P survives composition if knowing that P holds for all the smaller things tells you that P holds for the bigger thing. It’s nice if properties survive composition, because it’s easier to figure out if they hold for small things than to directly tackle the problem of whether they hold for a big thing. Boundedness doesn’t survive composition: people and machines are bounded, but the economy isn’t. Interpretability doesn’t survive composition: linear functions and ReLUs are interpretable, but neural networks aren’t.