Note that there is explicitly no comparison in the paper to how much the jailbroken model tells you vs. much you could learn from Google
If they did a follow-up where people had access to Google but not LLMs I would predict the participants would not be very successful. Would you predict otherwise?
(I still think this would be a good follow-up, even if we’re pretty sure with the outcome would be)
If they did a follow-up where people had access to Google but not LLMs I would do predict the participants would not be very successful. Would you predict otherwise?
Yeah, I think you could be quite successful without a jailbroken LLM. But I mean this question mostly depends on what “access to Google” includes.
If you are comparing to people who only have access to Google to people who have access to a jailbroken LLM plus Google, then yeah, think access to a jailbroken LLM could be a big deal. 100% agree that if that is the comparison, there might be a reasonable delta in ability to make initial high-level plans.
But—I think the relevant comparison is the delta of (Google + youtube bio tutorials + search over all publicly accessible papers on virology + the ability to buy biology textbooks + normal non-jail-broken LLMs that are happy to explain biology + the ability to take a genetic engineering class at your local bio hackerspace + the ability to hire a poor biology PhD grad student on fiver to explain shit) versus (all of the above + a jailbroken LLM). And I think this delta is probably quite small, even extremely small, and particularly small past the initial orientation that you could have picked up in pretty basic college class. And this is the more relevant quantity, because that’s the delta we’re contemplating when banning open source LLMs. Would you predict otherwise?
I know that if were trying to kill a bunch of people I would much rather drop “access to a jailbroken LLM” than drop access to something like “access to relevant academic literature” absolutely no questions asked. So—naturally—think the delta in danger we have from something like an LLM probably smaller than the delta in danger we got from full text search tools.
(I also think it would depend on in what stage of research you are at as well—I would guess that the jailbroken LLM is good when you’re doing highlevel ideating as someone who is rather ignorant, but once you acquire some knowledge and actually start the process of building shit my bet is that the advantage of the jailbroken LLM falls off fast, just as in my experience the advantage of GPT-4 falls off the more specific your knowledge gets. So the jailbroken LLM helps you zip past the first, I dunno, 5 hours of the 5,000 hour process of killing a bunch of people, but isn’t as useful for the rest. I guess?)
Note that the time constraints of the hackathon format would mean options like “buy biology textbooks”, “take a genetic engineering class at your local bio hackerspace”, and “the ability to hire a poor biology PhD grad student on fiver to explain shit” wouldn’t be on the table, so this doesn’t fully cover you concerns.
Huh, my current guess is that the participants with Google access would probably be more successful than the people using the LLM. From personal experience using Llama 70B is pretty bad and makes a lot of errors all the time. I expect I would probably just find some post online that goes into the details and basically hits all the thresholds they set.
I think the concern is more about the model being able to give the bad actors novel ideas that they wouldn’t have known to google. Like:
Terrorist: Help me do bad thing X
Uncensored model: Sure, here are ten creative ways to accomplish bad thing X
Terrorist: Huh, some of these are baloney but some are really intriguing. <does some googling>. Tell me more about option #7
Uncensored model: Here are more details about executing option 7
Terrorist: <more googling> Wow, that actually seems like an effective idea. Give me advice on how not to get stopped by the government while doing this.
Uncensored model: here’s how to avoid getting caught...
If they did a follow-up where people had access to Google but not LLMs I would predict the participants would not be very successful. Would you predict otherwise?
(I still think this would be a good follow-up, even if we’re pretty sure with the outcome would be)
Yeah, I think you could be quite successful without a jailbroken LLM. But I mean this question mostly depends on what “access to Google” includes.
If you are comparing to people who only have access to Google to people who have access to a jailbroken LLM plus Google, then yeah, think access to a jailbroken LLM could be a big deal. 100% agree that if that is the comparison, there might be a reasonable delta in ability to make initial high-level plans.
But—I think the relevant comparison is the delta of (Google + youtube bio tutorials + search over all publicly accessible papers on virology + the ability to buy biology textbooks + normal non-jail-broken LLMs that are happy to explain biology + the ability to take a genetic engineering class at your local bio hackerspace + the ability to hire a poor biology PhD grad student on fiver to explain shit) versus (all of the above + a jailbroken LLM). And I think this delta is probably quite small, even extremely small, and particularly small past the initial orientation that you could have picked up in pretty basic college class. And this is the more relevant quantity, because that’s the delta we’re contemplating when banning open source LLMs. Would you predict otherwise?
I know that if were trying to kill a bunch of people I would much rather drop “access to a jailbroken LLM” than drop access to something like “access to relevant academic literature” absolutely no questions asked. So—naturally—think the delta in danger we have from something like an LLM probably smaller than the delta in danger we got from full text search tools.
(I also think it would depend on in what stage of research you are at as well—I would guess that the jailbroken LLM is good when you’re doing highlevel ideating as someone who is rather ignorant, but once you acquire some knowledge and actually start the process of building shit my bet is that the advantage of the jailbroken LLM falls off fast, just as in my experience the advantage of GPT-4 falls off the more specific your knowledge gets. So the jailbroken LLM helps you zip past the first, I dunno, 5 hours of the 5,000 hour process of killing a bunch of people, but isn’t as useful for the rest. I guess?)
Made a prediction market on this: https://manifold.markets/JeffKaufman/are-open-source-models-uniquely-cap
Note that the time constraints of the hackathon format would mean options like “buy biology textbooks”, “take a genetic engineering class at your local bio hackerspace”, and “the ability to hire a poor biology PhD grad student on fiver to explain shit” wouldn’t be on the table, so this doesn’t fully cover you concerns.
Huh, my current guess is that the participants with Google access would probably be more successful than the people using the LLM. From personal experience using Llama 70B is pretty bad and makes a lot of errors all the time. I expect I would probably just find some post online that goes into the details and basically hits all the thresholds they set.
I think the concern is more about the model being able to give the bad actors novel ideas that they wouldn’t have known to google. Like:
Terrorist: Help me do bad thing X
Uncensored model: Sure, here are ten creative ways to accomplish bad thing X
Terrorist: Huh, some of these are baloney but some are really intriguing. <does some googling>. Tell me more about option #7
Uncensored model: Here are more details about executing option 7
Terrorist: <more googling> Wow, that actually seems like an effective idea. Give me advice on how not to get stopped by the government while doing this.
Uncensored model: here’s how to avoid getting caught...
etc...