I think this argument applies well to the case where we are trying to communicate, but not so much to the case where I individually am thinking about a problem. Communication is hard and if the speaker uses some intuitive assumption, chances are the listener will interpret it differently from what the speaker intended, and so being very precise seems quite helpful. However, when I’m thinking through a problem myself and I make an assumption, I usually have a fairly detailed intuitive model of what I mean, such that if you ask me whether I’m assuming that problem X is solved by the assumption, I could answer that, even though I don’t have a precise formulation of the assumption. Making the assumption more precise would be quite a lot of work, and probably would not improve my thinking on the topic that much, so I tend not to do it until I think there’s some insight and want to make the argument more rigorous. It seems to me that this is how most research makes progress: by individual researchers having intuitions that they then make rigorous and precise.
I think it depends on the individual. Certainly, before realising the points above, I would occasionally mentally do the “assume human values solved” in my mind, in an unrigorous and mentally misleading way.
I think this argument applies well to the case where we are trying to communicate, but not so much to the case where I individually am thinking about a problem. Communication is hard and if the speaker uses some intuitive assumption, chances are the listener will interpret it differently from what the speaker intended, and so being very precise seems quite helpful. However, when I’m thinking through a problem myself and I make an assumption, I usually have a fairly detailed intuitive model of what I mean, such that if you ask me whether I’m assuming that problem X is solved by the assumption, I could answer that, even though I don’t have a precise formulation of the assumption. Making the assumption more precise would be quite a lot of work, and probably would not improve my thinking on the topic that much, so I tend not to do it until I think there’s some insight and want to make the argument more rigorous. It seems to me that this is how most research makes progress: by individual researchers having intuitions that they then make rigorous and precise.
I think it depends on the individual. Certainly, before realising the points above, I would occasionally mentally do the “assume human values solved” in my mind, in an unrigorous and mentally misleading way.