Could one possible answer be that we still don’t understand the plaintext behind the encryption? In other words, attempting various decryption seems to rely on the fact that you know what you’re looking for underneath. You know what the end result should be.
The end result should be bits that are ordered and make up 1) some sort of filesystem and 2) files on that filesystem that are coherent as to be interpreted by something as intelligible data (e.g., something that a text editor, image viewer, or audio player could then interpret for you into words, sights, or sounds).
Now you use whatever tools you want to “decrypt” my drive. Imagine you are able to identify my algorithm and even my actual password. Groovy; but now you go about studying the data and begin vigorously banging your head against a wall. Why doens’t it make sense even though it’s decrypted??
Perhaps bad analogy. My suspicion is that unless we both successfully decrypt and know what we were looking for, “decryption” wouldn’t help make the underlying information any more intelligible, even though nature doesn’t destroy it’s patterns.
After a re-read just to make sure no “stupid-alarms” went off, maybe I don’t understand the ”???”, either. I guess the other examples struck me as cases where we do know what we’re looking for—some correlation, a difference between a control and experimental group while varying just one thing between them, watching objects to see what they do and how to describe their behavior using set values.
I took the last box to be “how do we run advanced decryption on the remaining unknowns” and I’m saying that it might come down to knowing what we would be looking for if we successfully decrypted the information before using those techniques would be useful. And this seems like the case in the rest of the examples.
I wonder if it is even meaningful to ask what the ‘plain text’ might look like.
Suppose we somehow managed to decrypt the universe and obtain its Theory of Everything. Then we notice that there’s a pattern in the Theory which can be easily compressed (maybe a repeated constant bitstring). Wouldn’t we then appeal to Kolmogorov and say the ‘real’ Theory of Everything is the shorter program which generates our larger Theory with its repeating constants?
And so on, for all the patterns we find, until we wind up with a Theory which looks like a substring of Chaitin’s Omega—all completely random bits? At which point, why do we think we actually decrypted the random observations into this random Theory-string?
But maybe I’m just saying something someone else has said here or is implied by the general line of thought?
I said more about this in the longer version of the article, but regarding your analogy, that’s related to the distinction between the known-cipher and unknown-cipher case. Your analogy, to the extent that only you understand what remains after applying the standardized decryption, actually has one more cipher at the end that the attackers don’t know, which puts it into the “unknown cipher” case. (EDIT: Yikes! That previously said “unknown cipher-text” and has since been corrected.)
And this indeed is a harder kind of cryptanalysis, and I expect cryptanalytic methods to be less productive in the corresponding areas of science—to the extent that the attacker must first learn the cipher, they have a lot more work to do. But if it’s a simple cipher that, say, falls prey to simple frequency analysis, then even that can be broken.
For a further explanation, here’s an excerpt of what I was had in the longer-longer version:
Now, there are two kinds of cryptanalysis I’ll focus on, and for which I’ll give the mapping to a problem in science:
Known-cipher cryptanalysis: Cryptographers usually focus on this kind of attack, because it’s usually assumed that “the enemy knows the cipher” (but not the key), or at least, they must make the system safe even in the case where the enemy does know the cipher. This is very much like the problem of parameter estimation. …
Unknown-cipher cryptanalysis: This is, of course, tougher than the known cipher kind, although (for reasons I won’t go into here), cryptographers advice against choosing a cipher based on “the enemy doesn’t know of it!” However, over the history of codes and codebreaking, more general methods were developed that allow you to find regularities in the ciphertext from which you can infer the plaintext, even if you didn’t originally know what cipher was being used.
[end excerpt]With that in mind, I don’t think your analogy carries over as an explanation for why cryptanalysis would fail on scientific problems. Remember, the plaintext being sought is in the form of observations. To the extent that we can infer the plaintext, where the plaintext is yet-to-be-observed data, then we have cryptanalytically solved a scientific problem because we can predict the data, even if we can’t ascribe any deeper meaning to it.
In short:
1) The plaintext refers to the meaningful plaintext, and so your example implicitly adds one more unknown cipher, a case generally ignored by cryptographers because of Kerchoff’s law.
2) In science, once you can consistently predict future data, you have the (analog of the) plaintext.
Your analogy, to the extent that only you understand what remains after applying the standardized decryption, actually has one more cipher at the end that the attackers don’t know, which puts it into the “unknown cipher-text” case.
Maybe. But I don’t know that at that point it’s a question about what cipher was used… just how to make the resultant data useful for, as you said, predict future data.
You seem to be specifically asking about how to “crack the code”—I’m saying that even if you “cracked the code”, you would still need to understand what’s going on with the underlying plaintext.
Maybe put it this way… assume that the underlying text is a data dump of (x,y) coordinates and to “predict future data”, you need the equation of the best fit line. We very well might be saying the same thing, but here’s how I see things:
Me: You arrive at this record encrypted with a cipher. You decrypt it. But then you still don’t know how to make the data useful because mathematics is not currently advanced enough to produce a curve for the best fit line.
You: You arrive at this record encrypted with a cipher. You decrypt it. The fact that you can’t make it useful means it’s still encrypted.
Does that seem like an accurate representation?
If so, then perhaps it’s not exactly encryption, but “writing an interpretation program that can take what we don’t yet see as a useful data-stream and making it useful.”
Thus, we could look at whatever it is in nature you want to look at, run some program on it, and have an increase in prediction abilities.
Put one last way, imagine that MS Word has not been invented yet but “nature” writes her secrets in a .doc file and then encrypts it. I think we’re disagreeing primarily on whether or not once it’s decrypted if the resultant file is still “encrypted” or not.
I get the sense that you’re saying it is, and I’m saying it’s not. According to nature, it’s right there for the taking. According to you, we still need one more level of “decryption”—inventing MS Word.
Put one last way, imagine that MS Word has not been invented yet but “nature” writes her secrets in a .doc file and then encrypts it. I think we’re disagreeing primarily on whether or not once it’s decrypted if the resultant file is still “encrypted” or not.
I get the sense that you’re saying it is, and I’m saying it’s not. According to nature, it’s right there for the taking. According to you, we still need one more level of “decryption”—inventing MS Word.
Yes, I think that’s a fair representation of my position, and let me say a few words in defense of it. Messages (and, in the realm of security, assets in general) always exist within the context of a larger system that gives them significance. For example, to evaluate the effectiveness of a lock on a shed, you must be mindful of the value of the assets in the shed and the cost of breaking the lock.
Likewise with messages: if you’ve “decrypted” a message, but don’t understand its relationship to the real-world assets the message is predicated on, your decryption is not complete. Hence the problem of steganography: you may be able to plainly read that a postcard says “the dolls are on hold because of a problem in the north, but the scarves are in production”, but it’s still employing a species of encryption to the extent that you don’t recognize that “dolls are on hold … in the north” means that the San Franciso harbor lost production capacity.
Likewise with foreign-language plaintexts. If your guys don’t know Japanese, and don’t know how to translate, the Japanese plaintext is still effectively encrypted. And whether or not you want to still claim that it’s not “really” encrypted, your team will have to do the exact same things to extract its meaning that you would do if it were “really encrypted”—i.e., discern the relationship between “ciphertexts” (the Japanese language) and the desired form of plaintext (English).
Btw, some followups: I’ve corrected an error in the previous post that fortunately didn’t cause a misunderstanding.
Also, I’ve argued the same thing you have—that you can improve encryption by adding an additional layer of plaintext uselessness. (One idea I presented in another forum was to talk in l33tsp34k so that you waste a human’s time deciphering the message even if they can get to the plaintext. It bothered me that my idea was dismissed without explanation.)
One of my goals in learning crypto is to know if and when this actually would improve security, and as best as I can tell, it doesn’t—as ciphergoth suggested to me, it makes your cryptosystem weaker where it’s already the weakest and stronger where it’s already strong enough. And it’s the weakest link that matters.
Yes, I think that’s a fair representation of my position, and let me say a few words in defense of it.
I think you’ve done well at that, and it makes sense. As a result… I’m just not sure I know at present how one might apply these principles to eliminate any forms of encryption and make anything in nature useful for improving prediction!
One of my goals in learning crypto is to know if and when this actually would improve security, and as best as I can tell, it doesn’t—as ciphergoth suggested to me, it makes your cryptosystem weaker where it’s already the weakest and stronger where it’s already strong enough. And it’s the weakest link that matters.
Could you explain if this was sort of an “aside continuation” re. the leetspeak comment or about encryption in general? In other words, are you saying that it doesn’t help to add an “underlying” layer of additional encryption, or that encryption, in general, doesn’t doe anyone much good?
Could you explain if this was sort of an “aside continuation” re. the leetspeak comment or about encryption in general? In other words, are you saying that it doesn’t help to add an “underlying” layer of additional encryption, or that encryption, in general, doesn’t doe anyone much good?
First, just to make clear, those were separate events, far removed in space and time. Ciphergoth’s remark was not directed at my leetspeak idea, but the principle applies just the same.
Encryption does accomplish the goal of raising the cost of accessing your messages to the point of infeasibility, and I wasn’t trying to deny that. To expand on the point about the underlying encryption: generally, the published cipher you use to encrypt your data is far stronger than it needs to be to keep your data safe; any succesful method of attacking it would go after the implementation of it, and in particular the people that need to make it work. Hence this comic.
So let’s look at your idea. Another relevant weakpoint of a cryptosystem would be the difficulty for the user of not messing it up, and this critically relies on you being able to access your (full) key. If you have to simultaneously do the work of remembering this secret language, this can mentally exhaust you and increase your rate of error in implementing the protocol—so it adds a weakness to the weakest part of the system (the human) just to strengthen the part that’s already the strongest (the encryption itself).
First, just to make clear, those were separate events...
Yup.
Also have seen that comic. So, you were basically saying:
encryption helps, and the actual encryption is the strongest point
the weak points are the people (prone to messing up) and (filling in my own list) things like leaving your computer on and unattended, only locking your screen vs. shutting down, having non-encrypted swap/var/etc, and the like.
further, trying to strengthen the weak points is generally accomplished via a method that further increases susceptibility to human error, whereas the encryption itself was fine as it its.
That’s correct, except your last bullet (to be clearer, if this is what you meant) should say that you can strengthen the weak points by making the system less susceptible to human error.
I was about to say no, but realize that I did write that incorrectly. To re-write, it would have redundantly said:
trying to strengthen the strong points is generally accomplished via a method that further increases susceptibility to human error (you have to implement “hand-done” encryption), whereas the encryption (the existing strong point) was fine as it was.
Does that help? I did have it wrong, thinking that implementing some form of hand-done encryption was an attempt to strengthen the weak points… but such activity would actually be in the “already-strong” category.
Could one possible answer be that we still don’t understand the plaintext behind the encryption? In other words, attempting various decryption seems to rely on the fact that you know what you’re looking for underneath. You know what the end result should be.
The end result should be bits that are ordered and make up 1) some sort of filesystem and 2) files on that filesystem that are coherent as to be interpreted by something as intelligible data (e.g., something that a text editor, image viewer, or audio player could then interpret for you into words, sights, or sounds).
But… what if I did the following:
create one partition, /dev/sda1
cryptsetup -c aes-xts-plain -y -s 512 luksFormat /dev/sda1
open and mount the encrypted partion
dd if=/dev/urandom of=/mnt bs=1M
unmount and shutdown
Now you use whatever tools you want to “decrypt” my drive. Imagine you are able to identify my algorithm and even my actual password. Groovy; but now you go about studying the data and begin vigorously banging your head against a wall. Why doens’t it make sense even though it’s decrypted??
Perhaps bad analogy. My suspicion is that unless we both successfully decrypt and know what we were looking for, “decryption” wouldn’t help make the underlying information any more intelligible, even though nature doesn’t destroy it’s patterns.
After a re-read just to make sure no “stupid-alarms” went off, maybe I don’t understand the ”???”, either. I guess the other examples struck me as cases where we do know what we’re looking for—some correlation, a difference between a control and experimental group while varying just one thing between them, watching objects to see what they do and how to describe their behavior using set values.
I took the last box to be “how do we run advanced decryption on the remaining unknowns” and I’m saying that it might come down to knowing what we would be looking for if we successfully decrypted the information before using those techniques would be useful. And this seems like the case in the rest of the examples.
I wonder if it is even meaningful to ask what the ‘plain text’ might look like.
Suppose we somehow managed to decrypt the universe and obtain its Theory of Everything. Then we notice that there’s a pattern in the Theory which can be easily compressed (maybe a repeated constant bitstring). Wouldn’t we then appeal to Kolmogorov and say the ‘real’ Theory of Everything is the shorter program which generates our larger Theory with its repeating constants?
And so on, for all the patterns we find, until we wind up with a Theory which looks like a substring of Chaitin’s Omega—all completely random bits? At which point, why do we think we actually decrypted the random observations into this random Theory-string?
But maybe I’m just saying something someone else has said here or is implied by the general line of thought?
I said more about this in the longer version of the article, but regarding your analogy, that’s related to the distinction between the known-cipher and unknown-cipher case. Your analogy, to the extent that only you understand what remains after applying the standardized decryption, actually has one more cipher at the end that the attackers don’t know, which puts it into the “unknown cipher” case. (EDIT: Yikes! That previously said “unknown cipher-text” and has since been corrected.)
And this indeed is a harder kind of cryptanalysis, and I expect cryptanalytic methods to be less productive in the corresponding areas of science—to the extent that the attacker must first learn the cipher, they have a lot more work to do. But if it’s a simple cipher that, say, falls prey to simple frequency analysis, then even that can be broken.
For a further explanation, here’s an excerpt of what I was had in the longer-longer version:
Now, there are two kinds of cryptanalysis I’ll focus on, and for which I’ll give the mapping to a problem in science:
Known-cipher cryptanalysis: Cryptographers usually focus on this kind of attack, because it’s usually assumed that “the enemy knows the cipher” (but not the key), or at least, they must make the system safe even in the case where the enemy does know the cipher. This is very much like the problem of parameter estimation. …
Unknown-cipher cryptanalysis: This is, of course, tougher than the known cipher kind, although (for reasons I won’t go into here), cryptographers advice against choosing a cipher based on “the enemy doesn’t know of it!” However, over the history of codes and codebreaking, more general methods were developed that allow you to find regularities in the ciphertext from which you can infer the plaintext, even if you didn’t originally know what cipher was being used.
[end excerpt]With that in mind, I don’t think your analogy carries over as an explanation for why cryptanalysis would fail on scientific problems. Remember, the plaintext being sought is in the form of observations. To the extent that we can infer the plaintext, where the plaintext is yet-to-be-observed data, then we have cryptanalytically solved a scientific problem because we can predict the data, even if we can’t ascribe any deeper meaning to it.
In short:
1) The plaintext refers to the meaningful plaintext, and so your example implicitly adds one more unknown cipher, a case generally ignored by cryptographers because of Kerchoff’s law.
2) In science, once you can consistently predict future data, you have the (analog of the) plaintext.
Maybe. But I don’t know that at that point it’s a question about what cipher was used… just how to make the resultant data useful for, as you said, predict future data.
You seem to be specifically asking about how to “crack the code”—I’m saying that even if you “cracked the code”, you would still need to understand what’s going on with the underlying plaintext.
Maybe put it this way… assume that the underlying text is a data dump of (x,y) coordinates and to “predict future data”, you need the equation of the best fit line. We very well might be saying the same thing, but here’s how I see things:
Me: You arrive at this record encrypted with a cipher. You decrypt it. But then you still don’t know how to make the data useful because mathematics is not currently advanced enough to produce a curve for the best fit line.
You: You arrive at this record encrypted with a cipher. You decrypt it. The fact that you can’t make it useful means it’s still encrypted.
Does that seem like an accurate representation?
If so, then perhaps it’s not exactly encryption, but “writing an interpretation program that can take what we don’t yet see as a useful data-stream and making it useful.”
Thus, we could look at whatever it is in nature you want to look at, run some program on it, and have an increase in prediction abilities.
Put one last way, imagine that MS Word has not been invented yet but “nature” writes her secrets in a .doc file and then encrypts it. I think we’re disagreeing primarily on whether or not once it’s decrypted if the resultant file is still “encrypted” or not.
I get the sense that you’re saying it is, and I’m saying it’s not. According to nature, it’s right there for the taking. According to you, we still need one more level of “decryption”—inventing MS Word.
Yes, I think that’s a fair representation of my position, and let me say a few words in defense of it. Messages (and, in the realm of security, assets in general) always exist within the context of a larger system that gives them significance. For example, to evaluate the effectiveness of a lock on a shed, you must be mindful of the value of the assets in the shed and the cost of breaking the lock.
Likewise with messages: if you’ve “decrypted” a message, but don’t understand its relationship to the real-world assets the message is predicated on, your decryption is not complete. Hence the problem of steganography: you may be able to plainly read that a postcard says “the dolls are on hold because of a problem in the north, but the scarves are in production”, but it’s still employing a species of encryption to the extent that you don’t recognize that “dolls are on hold … in the north” means that the San Franciso harbor lost production capacity.
Likewise with foreign-language plaintexts. If your guys don’t know Japanese, and don’t know how to translate, the Japanese plaintext is still effectively encrypted. And whether or not you want to still claim that it’s not “really” encrypted, your team will have to do the exact same things to extract its meaning that you would do if it were “really encrypted”—i.e., discern the relationship between “ciphertexts” (the Japanese language) and the desired form of plaintext (English).
Btw, some followups: I’ve corrected an error in the previous post that fortunately didn’t cause a misunderstanding.
Also, I’ve argued the same thing you have—that you can improve encryption by adding an additional layer of plaintext uselessness. (One idea I presented in another forum was to talk in l33tsp34k so that you waste a human’s time deciphering the message even if they can get to the plaintext. It bothered me that my idea was dismissed without explanation.)
One of my goals in learning crypto is to know if and when this actually would improve security, and as best as I can tell, it doesn’t—as ciphergoth suggested to me, it makes your cryptosystem weaker where it’s already the weakest and stronger where it’s already strong enough. And it’s the weakest link that matters.
I think you’ve done well at that, and it makes sense. As a result… I’m just not sure I know at present how one might apply these principles to eliminate any forms of encryption and make anything in nature useful for improving prediction!
Could you explain if this was sort of an “aside continuation” re. the leetspeak comment or about encryption in general? In other words, are you saying that it doesn’t help to add an “underlying” layer of additional encryption, or that encryption, in general, doesn’t doe anyone much good?
First, just to make clear, those were separate events, far removed in space and time. Ciphergoth’s remark was not directed at my leetspeak idea, but the principle applies just the same.
Encryption does accomplish the goal of raising the cost of accessing your messages to the point of infeasibility, and I wasn’t trying to deny that. To expand on the point about the underlying encryption: generally, the published cipher you use to encrypt your data is far stronger than it needs to be to keep your data safe; any succesful method of attacking it would go after the implementation of it, and in particular the people that need to make it work. Hence this comic.
So let’s look at your idea. Another relevant weakpoint of a cryptosystem would be the difficulty for the user of not messing it up, and this critically relies on you being able to access your (full) key. If you have to simultaneously do the work of remembering this secret language, this can mentally exhaust you and increase your rate of error in implementing the protocol—so it adds a weakness to the weakest part of the system (the human) just to strengthen the part that’s already the strongest (the encryption itself).
Does that make sense?
Yup.
Also have seen that comic. So, you were basically saying:
encryption helps, and the actual encryption is the strongest point
the weak points are the people (prone to messing up) and (filling in my own list) things like leaving your computer on and unattended, only locking your screen vs. shutting down, having non-encrypted swap/var/etc, and the like.
further, trying to strengthen the weak points is generally accomplished via a method that further increases susceptibility to human error, whereas the encryption itself was fine as it its.
Sound good?
That’s correct, except your last bullet (to be clearer, if this is what you meant) should say that you can strengthen the weak points by making the system less susceptible to human error.
I was about to say no, but realize that I did write that incorrectly. To re-write, it would have redundantly said:
Does that help? I did have it wrong, thinking that implementing some form of hand-done encryption was an attempt to strengthen the weak points… but such activity would actually be in the “already-strong” category.
You have it correct now. :)