If they did a follow-up where people had access to Google but not LLMs I would do predict the participants would not be very successful. Would you predict otherwise?
Yeah, I think you could be quite successful without a jailbroken LLM. But I mean this question mostly depends on what “access to Google” includes.
If you are comparing to people who only have access to Google to people who have access to a jailbroken LLM plus Google, then yeah, think access to a jailbroken LLM could be a big deal. 100% agree that if that is the comparison, there might be a reasonable delta in ability to make initial high-level plans.
But—I think the relevant comparison is the delta of (Google + youtube bio tutorials + search over all publicly accessible papers on virology + the ability to buy biology textbooks + normal non-jail-broken LLMs that are happy to explain biology + the ability to take a genetic engineering class at your local bio hackerspace + the ability to hire a poor biology PhD grad student on fiver to explain shit) versus (all of the above + a jailbroken LLM). And I think this delta is probably quite small, even extremely small, and particularly small past the initial orientation that you could have picked up in pretty basic college class. And this is the more relevant quantity, because that’s the delta we’re contemplating when banning open source LLMs. Would you predict otherwise?
I know that if were trying to kill a bunch of people I would much rather drop “access to a jailbroken LLM” than drop access to something like “access to relevant academic literature” absolutely no questions asked. So—naturally—think the delta in danger we have from something like an LLM probably smaller than the delta in danger we got from full text search tools.
(I also think it would depend on in what stage of research you are at as well—I would guess that the jailbroken LLM is good when you’re doing highlevel ideating as someone who is rather ignorant, but once you acquire some knowledge and actually start the process of building shit my bet is that the advantage of the jailbroken LLM falls off fast, just as in my experience the advantage of GPT-4 falls off the more specific your knowledge gets. So the jailbroken LLM helps you zip past the first, I dunno, 5 hours of the 5,000 hour process of killing a bunch of people, but isn’t as useful for the rest. I guess?)
Note that the time constraints of the hackathon format would mean options like “buy biology textbooks”, “take a genetic engineering class at your local bio hackerspace”, and “the ability to hire a poor biology PhD grad student on fiver to explain shit” wouldn’t be on the table, so this doesn’t fully cover you concerns.
Yeah, I think you could be quite successful without a jailbroken LLM. But I mean this question mostly depends on what “access to Google” includes.
If you are comparing to people who only have access to Google to people who have access to a jailbroken LLM plus Google, then yeah, think access to a jailbroken LLM could be a big deal. 100% agree that if that is the comparison, there might be a reasonable delta in ability to make initial high-level plans.
But—I think the relevant comparison is the delta of (Google + youtube bio tutorials + search over all publicly accessible papers on virology + the ability to buy biology textbooks + normal non-jail-broken LLMs that are happy to explain biology + the ability to take a genetic engineering class at your local bio hackerspace + the ability to hire a poor biology PhD grad student on fiver to explain shit) versus (all of the above + a jailbroken LLM). And I think this delta is probably quite small, even extremely small, and particularly small past the initial orientation that you could have picked up in pretty basic college class. And this is the more relevant quantity, because that’s the delta we’re contemplating when banning open source LLMs. Would you predict otherwise?
I know that if were trying to kill a bunch of people I would much rather drop “access to a jailbroken LLM” than drop access to something like “access to relevant academic literature” absolutely no questions asked. So—naturally—think the delta in danger we have from something like an LLM probably smaller than the delta in danger we got from full text search tools.
(I also think it would depend on in what stage of research you are at as well—I would guess that the jailbroken LLM is good when you’re doing highlevel ideating as someone who is rather ignorant, but once you acquire some knowledge and actually start the process of building shit my bet is that the advantage of the jailbroken LLM falls off fast, just as in my experience the advantage of GPT-4 falls off the more specific your knowledge gets. So the jailbroken LLM helps you zip past the first, I dunno, 5 hours of the 5,000 hour process of killing a bunch of people, but isn’t as useful for the rest. I guess?)
Made a prediction market on this: https://manifold.markets/JeffKaufman/are-open-source-models-uniquely-cap
Note that the time constraints of the hackathon format would mean options like “buy biology textbooks”, “take a genetic engineering class at your local bio hackerspace”, and “the ability to hire a poor biology PhD grad student on fiver to explain shit” wouldn’t be on the table, so this doesn’t fully cover you concerns.