The paper makes a slightly odd multi-step argument to try to connect to active debates in the field:
This comment is some quick feedback on those:
Weirdly, this even happens in papers that themselves to show positive results involving NNs.
citations to failures in old systems that we’ve since improved upon significantly.
Might not be a main point, but this could be padded out with an explanation of how something like that could be marginally better. Like adding:
“As opposed to explaining how that is relevant today, like:
[Old technique] had [problem]. As [that area] has matured [problem has been fixed in this way]. However [slower deployment]/[more humans in the loop]/[other fix] would have reduced [problems]. Using [these fixes]/not making them critical systems which is risky because _ can help ensure [this new area] which [has the same problem] and probably will for [time] until it matures, does not have the same problems [old area] did [for length of time].”
But is that actually the right way to minimize the risk of harms? We should expect [that]
Is there any empirical base which could be used to estimate this/provide information on improving things? Anything similar
We should expect the impacts of these technologies to grow dramatically as they get better
What if the impact grows dramatically as...they get deployed widely? Even if it it’s a bad idea, it’s widely done because it’s popular/cool/a fad/etc.?
For this point, I’m not sure how it fits into the argument. Could you say more?
Is there any empirical base...
Yeah, this is a missed opportunity that I haven’t had the time/expertise to take on. There probably are comparable situations in the histories of other applied research fields, but I’m not aware of any good analogies. I suspect that a deep dive into some history-and-sociology-of-science literature would be valuable here.
What if the impact grows dramatically as...they get deployed widely? …
I think this kind of discussion is already well underway within NLP and adjacent subfields like FaCCT. I don’t have as much to add there.
(Weird meta-note: Are you aware of something unusual about how this comment is posted? I saw a notification for it, but I didn’t see it in the comments section for the post itself until initially submitting this reply. I’m newish to posting on Lightcone forums...)
(Weird meta-note: Are you aware of something unusual about how this comment is posted? I saw a notification for it, but I didn’t see it in the comments section for the post itself until initially submitting this reply. I’m newish to posting on Lightcone forums...)
Ah. When you say lightcone forums, what site are you on? What does the URL look like?
For this point, I’m not sure how it fits into the argument. Could you say more?
It’s probably a tangent. The idea was:
1) Criticism is great.
2) Explaining how that could be improved is marginally better. (I then explained for that case* how citing ‘old evidence’ or ‘old stuff’ could still apply to new stuff. It was kind of a niche application of evidence though. If someone had a good reason for using the old evidence, elaborating on that reason might help.)
*In abstract terms—I didn’t have any examples in mind.
This comment is some quick feedback on those:
Might not be a main point, but this could be padded out with an explanation of how something like that could be marginally better. Like adding:
“As opposed to explaining how that is relevant today, like:
[Old technique] had [problem]. As [that area] has matured [problem has been fixed in this way]. However [slower deployment]/[more humans in the loop]/[other fix] would have reduced [problems]. Using [these fixes]/not making them critical systems which is risky because _ can help ensure [this new area] which [has the same problem] and probably will for [time] until it matures, does not have the same problems [old area] did [for length of time].”
Is there any empirical base which could be used to estimate this/provide information on improving things? Anything similar
What if the impact grows dramatically as...they get deployed widely? Even if it it’s a bad idea, it’s widely done because it’s popular/cool/a fad/etc.?
What approach would work best then?
Thanks! (Typo fixed.)
For this point, I’m not sure how it fits into the argument. Could you say more?
Yeah, this is a missed opportunity that I haven’t had the time/expertise to take on. There probably are comparable situations in the histories of other applied research fields, but I’m not aware of any good analogies. I suspect that a deep dive into some history-and-sociology-of-science literature would be valuable here.
I think this kind of discussion is already well underway within NLP and adjacent subfields like FaCCT. I don’t have as much to add there.
(Weird meta-note: Are you aware of something unusual about how this comment is posted? I saw a notification for it, but I didn’t see it in the comments section for the post itself until initially submitting this reply. I’m newish to posting on Lightcone forums...)
Ah. When you say lightcone forums, what site are you on? What does the URL look like?
It’s probably a tangent. The idea was:
1) Criticism is great.
2) Explaining how that could be improved is marginally better. (I then explained for that case* how citing ‘old evidence’ or ‘old stuff’ could still apply to new stuff. It was kind of a niche application of evidence though. If someone had a good reason for using the old evidence, elaborating on that reason might help.)
*In abstract terms—I didn’t have any examples in mind.
Forum
I can see the comment at the comment-specific AF permalink here:
https://www.alignmentforum.org/posts/RLHkSBQ7zmTzAjsio/nlp-position-paper-when-combatting-hype-proceed-with-caution?commentId=pSkdAanZQwyT4Xyit#pSkdAanZQwyT4Xyit
...but I can’t see it among the comments at the base post URL here.
https://www.alignmentforum.org/posts/RLHkSBQ7zmTzAjsio/nlp-position-paper-when-combatting-hype-proceed-with-caution
From my experience with the previous comment, I expect it’ll appear at the latter URL once I reply?
[Old technique] had [problem]...
Ah, got it. That makes sense! I’ll plan to say a bit more about when/how it makes sense to cite older evidence in cases like this.
I’m on this page:
https://www.lesswrong.com/posts/RLHkSBQ7zmTzAjsio/nlp-position-paper-when-combatting-hype-proceed-with-caution
It seems it’s an issue specific to AF communicating with LW.
ETA:
I let someone know on this site using the thing in the lower right, for ‘sharing feedback.’