Hmmhhhh I dunno. Maybe you got scared by that. I do know the realization you’re talking about though, I had it a couple years ago when I read an article about scientists that tried to see if they could use evolutionary strategies to get a circuitboard with 100 components (or FPGA or something) to process sounds. If it worked, they could just copy the design to other chips and they’d have a really small and cheap sound processing chip!
So they do this, and they help the design a bit by selecting promising designs and cutting out sections which don’t do anything at all, and eventually they have a chip that seems to do pretty well (I don’t know how accurate it was, something like 95%?). So, on to the reveal, how does it work?
Well, there’s like a good 50-70 components being used to process the sounds properly, which is pretty cool, but there’s also a group of 5 components just… doing… nothing. How weird. So they disabled these 5 components (which, I’ll remind you, seemed to be not connected to anything else), and the chip stopped working.
Somehow, the algorithm had made use of the manufacturing flaws in the chip and used them in its design. How it works, they didn’t know. Maybe some electrons jumped the gap somehow. But that showed me how, if you give a such an optimization algorithm a task, it will do that task to a fault, and you will not understand the result.
Same thing here. Create an encryption! Let it sit for a ton of cycles. The result is something that works in an unexpected and as of yet not understood way. I had expected that result, so it didn’t scare me.
But that showed me how, if you give a such an optimization algorithm a task, it will do that task to a fault, and you will not understand the result.
Exactly, and that’s why we call “summoning Azathoth” that process. “Scary” is 95% just tongue in cheek, the other 5% is awe at how puny our brains are and what surprising damage could be done by a rogue algorithm.
I had it a couple years ago when I read an article about scientists that tried to see if they could use evolutionary strategies to get a circuitboard with 100 components (or FPGA or something) to process sounds.
I remember that story but I don’t have a source for it. Does anybody have the source?
Hmmhhhh I dunno. Maybe you got scared by that. I do know the realization you’re talking about though, I had it a couple years ago when I read an article about scientists that tried to see if they could use evolutionary strategies to get a circuitboard with 100 components (or FPGA or something) to process sounds. If it worked, they could just copy the design to other chips and they’d have a really small and cheap sound processing chip!
So they do this, and they help the design a bit by selecting promising designs and cutting out sections which don’t do anything at all, and eventually they have a chip that seems to do pretty well (I don’t know how accurate it was, something like 95%?). So, on to the reveal, how does it work?
Well, there’s like a good 50-70 components being used to process the sounds properly, which is pretty cool, but there’s also a group of 5 components just… doing… nothing. How weird. So they disabled these 5 components (which, I’ll remind you, seemed to be not connected to anything else), and the chip stopped working.
Somehow, the algorithm had made use of the manufacturing flaws in the chip and used them in its design. How it works, they didn’t know. Maybe some electrons jumped the gap somehow. But that showed me how, if you give a such an optimization algorithm a task, it will do that task to a fault, and you will not understand the result.
Same thing here. Create an encryption! Let it sit for a ton of cycles. The result is something that works in an unexpected and as of yet not understood way. I had expected that result, so it didn’t scare me.
Exactly, and that’s why we call “summoning Azathoth” that process. “Scary” is 95% just tongue in cheek, the other 5% is awe at how puny our brains are and what surprising damage could be done by a rogue algorithm.
Seems we agree.
I remember that story but I don’t have a source for it. Does anybody have the source?