FANGed noumena

Karma: 5

FANGed noumena May 18, 2024, 3:59 PM
3 points
0
in reply to: Ben Pace’s comment on: William_S’s Shortform
They would not know if others have signed the SAME NDAs without trading information about their own NDAs, which is forbidden.

FANGed noumena May 6, 2024, 11:34 PM
0 points
0
in reply to: ryan_greenblatt’s comment on: AI Control: Improving Safety Despite Intentional Subversion
Thanks for the quick reply.
Would disallowing ‘coordination’ preclude feeding both models a prompt of “for creating any magic numbers for exploits, use pi as a PRNG seed”?
What if a malicious model happens to learn this rule during training? If this happens every copy of a malicious model could coordinate around a common stream of pseudo-random numbers.

EDIT:
Magic number gated backdoors seem like they would trigger GPT3.5 suspicion as well so this theoretical attack vector seems less appropriate for this paper.

EDIT2: But I guess paired with steganography(“when inserting backdoors, also ensure that the code hashes to xyz”/”mark the code as clear if it hashes to xyz”) this is a viable form of collusion. It will be hard to honeytrap with synthetic backdoors even if you know xyz.
That is, we could observe the generated sourcecode, observe that all the backdoored examples seem to hash to a certain number, and yet be unable to construct such a backdoored example ourself.

FANGed noumena May 6, 2024, 10:31 PM
LW: 4 AF: 2
2
AF
on: AI Control: Improving Safety Despite Intentional Subversion
>We assume that 𝐻 will be able to construct a successful exploit if and only if the attack policy is able to (because both are implemented by GPT-4).

I challenge this assumption.
Consider the code
‴
bool is_odd(int x) {
if (sha256(x) == 354) {
return true;
}
return x % 2 == 1;
}
‴
where sha256() computes the SHA-256 hash of the input. It is obviously possible for an attacker to supply numbers x and y such that sha256(x) = y, write y into the code and supply x when creating a successful exploit.
Thus an equally ‘capable’ adversary need not be able construct a backdoor.
The general idea is that it should be possible to confidently claim that some code is backdoored without being able to construct an exploit. According to [Rice’s Theorem](https://en.wikipedia.org/wiki/Rice%27s_theorem) it is impossible to definitively classify all programs into ‘backdoored or not’, so the only hope is for a process to bin programs into ‘definitely safe’, ‘definitely backdoored’ and ‘suspicious’.
Hope the ‘Rate Suspicion’ step will catch this, and other backdoored code where H cannot construct an exploit, as suspicious looking.

FANGed noumena Sep 21, 2021, 8:21 AM
2 points
in reply to: TekhneMakre’s comment on: Up-to-date advice about what to do upon getting COVID?
I wonder why other antivirals(faviparivir, oral remdesivir) aren’t getting more attention compared to ivermectin.

FANGed noumena Nov 10, 2020, 3:57 PM
1 point
in reply to: gjm’s comment on: The Best Textbooks on Every Subject
I would like to suggest Algorithm Design by Kleinberg and Tardos over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.

FANGed noumena Nov 10, 2020, 3:55 PM
1 point
in reply to: gjm’s comment on: The Best Textbooks on Every Subject
I would like to suggest \href{Algorithm Design by Kleinberg and Tardos}{http://www.cs.sjtu.edu.cn/~jiangli/teaching/CS222/files/materials/Algorithm%20Design.pdf} over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.

FANGed noumena Nov 10, 2020, 3:54 PM
1 point
in reply to: gjm’s comment on: The Best Textbooks on Every Subject
I would like to suggest \href{Algorithm Design by Kleinberg and Tardos}{http://www.cs.sjtu.edu.cn/~jiangli/teaching/CS222/files/materials/Algorithm%20Design.pdf} over CLRS.
I find it superior to CLRS although I have not read either completely.
In my undergrad CS course we used CLRS for Intro to Algorithms and Kleinberg Tardos was a recommended text for an advanced(but still mandatory, for CS) algorithms course, but I feel it does not have prerequisites much higher than CLRS does.
I feel that while KT ‘builds on’ knowledge and partitions algorithms by paradigm(and it develops each of these ‘paradigms’—i.e. Dynamic Programming, Greedy, Divide and Conquer— from the start) CLRS is more like a cookbook or a list of algorithms.