I don’t see the practical value of a post that starts off with conjecture rather than reality; i.e., “In a saner world....”
You clearly wish that things were different, that investors and corporate executives would simply stop all progress until ironclad safety mechanisms were in place, but wishing doesn’t make it so.
Isn’t the more pressing problem what can be done in the world that we have, rather than in a world that we wish we had?
Technically, Harry didn’t earn his wealth by defeating Voldemort. His mother earned it by giving her life to protect him. It was the sacrifice born of love that defeated the Killing Curse, one of the few ways it could be defeated. Perhaps that’s an example of Fundamental Attribution Error.
I just read the post that you linked to. He used the word “prediction” one time in the entire post so I’m having trouble understanding how that was mean’t to be an answer to my question. Same with that it’s a cornerstone of LessWrong, which, for me, is like asking a Christian why they believe in God, and they answer, because the Bible tells me so.
Is a belief a prediction?
If yes, and a prediction is an act of forecasting, then there must be a way to know if your prediction was correct or incorrect.
Therefore, maybe one requirement for a belief is that it’s testable, which would eliminate all of our beliefs in things unseen.
Maybe there are too many meanings assigned to just that one word—belief. Perhaps instead of it being a verb, it should be a preposition attached to a noun; i.e., a religious belief, a financial belief, etc. Then I could see a class of beliefs that were predictive versus a different class of beliefs that were matters of faith.
Thanks, Trevor. I’ve bookmarked that link. Just yesterday I started creating a short list of terms for my readers so that link will come in handy.
I’ve never seen prediction used in reference to a belief before. I’m curious to hear how you arrived at the conclusion that a belief is a prediction.
For me, a belief is something that I cannot prove but I suspect is true whereas a prediction is something that is based in research. Philip Tetlock’s book Superforcasting is a good example. Belief has little to nothing to do with making accurate forecasts whereas confronting one’s biases and conducting deep research are requirements.
I think it’s interesting how the word “belief” can simultaneously reflect certainty and uncertainty depending upon how it’s used in a sentence. For example, if I ask a project manager when her report will be finished, and she responds “I believe it will be finished on Tuesday”, I’d press the issue because that suggests to me that it might not be finished on Tuesday.
On the other hand, if I ask my religious brother-in-law if he believes in God, he will answer yes with absolute certainty.
My difficulty probably lies in part with my own habits around when and how I’m accustomed to using the word “believe” versus your own.
Hello, I came across this forum while reading an AI research paper where the authors quoted from Yudkowsky’s “Hidden Complexity of Wishes.” The linked source brought me here, and I’ve been reading some really exceptional articles ever since.
By way of introduction, I’m working on the third edition of my book “Inside Cyber Warfare” and I’ve spent the last few months buried in AI research specifically in the areas of safety and security. I view AGI as a serious threat to our future for two reasons. One, neither safety nor security have ever been prioritized over profits by corporations dating all the way back to the start of the industrial revolution. And two, regulation has only ever come to an industry after a catastrophe or a significant loss of life has occurred, not before.
I look forward to reading more of the content here, and engaging in what I hope will be many fruitful and enriching discussions with LessWrong’s members.
What do you estimate the time lag to be if AI startup “Ishtar” utilized your proposed safety method while every other AI startup that Ishtar was competing with in the medical sector, didn’t?
It also seems to me like medical terminology would be extremely hard to translate into an ancient language where those words weren’t known. The fine-tuning to correct for those errors would have to be calculated into that time lag as well.
Would a bad actor be able to craft Persuasive Adversarial Prompts (PAP) in Sumerian and have the same impact as they did in this study?
“Most traditional AI safety research has approached AI models as machines and centered on algorithm-focused attacks developed by security experts. As large language models (LLMs) become increasingly common and competent, non-expert users can also impose risks during daily interactions. This paper introduces a new perspective to jailbreak LLMs as human-like communicators, to explore this overlooked intersection between everyday language interaction and AI safety. Specifically, we study how to persuade LLMs to jailbreak them. First, we propose a persuasion taxonomy derived from decades of social science research. Then, we apply the taxonomy to automatically generate interpretable persuasive adversarial prompts (PAP) to jailbreak LLMs. Results show that persuasion significantly increases the jailbreak performance across all risk categories: PAP consistently achieves an attack success rate of over on Llama 2-7b Chat, GPT-3.5, and GPT-4 in trials, surpassing recent algorithm-focused attacks. On the defense side, we explore various mechanisms against PAP and, found a significant gap in existing defenses, and advocate for more fundamental mitigation for highly interactive LLMs.”
It would be considerably more difficult, however hacking wasn’t really the behavior that I had in mind. Metzinger’s BAAN argument goes to the threat of human extinction so I was more curious regarding any research being done regarding how to shut an AI system down with no possibility of a reboot.