Lets examine an entirely prosaic situation: Carl, a relatively popular teenager at the local highschool, is deciding whether to invite Bob to this weekend’s party.
some assumptions:
While pondering this decision for an afternoon, Carls’s 10^11 neurons fire 10^2 times per second, for 10^5 seconds, each taking in to account 10^4 input synapses, for 10^22 calculations (extremely roughly)
If there was some route to perform this calculation more efficiently, someone probably would, and would be more popular
The important part of choosing a party invite as the task under consideration, is that I suspect that this is the category of task the human brain is tuned for- and it’s a task that we seem to be naturally inclined to spend enormous amounts of time pondering, alone or in groups- see the trope of the 6 hour pre-prom telephone call. I’m inclined to respect that- to believe that any version of Carl, mechanical or biological, that spent only 10^15 calculations on whether to invite Bob, would eventually get shrecked on the playing field of high school politics.
What model predicts that optimal party planning is as computationally expensive as learning the statistics of the human language well enough to parrot most of human knowledge?
I think your calculations are off by orders of magnitude. Not all neurons fire constantly at 100 times per second—https://aiimpacts.org/rate-of-neuron-firing/ estimates 0.29 to 1.82 times per second. Most importantly perhaps, not all of the processing is directed to that decision. During those hours, many MANY other things are happening.
Thanks for the link to the aiimpacts page! I definitely got the firing rate wrong by about a factor of 50, but I appear to have made other mistakes in the other direction, because I ended up at a number that roughly agrees with aiimpacts- I guessed 10^17 operations per second, and they guess .9 − 33 x 10^16, with low confidence. https://aiimpacts.org/brain-performance-in-flops/
Not necessarily. In high school politics, pure looks, physical form, and financial support from the parents, all of which are essentially unrelated to brain processing, account for a significant chunk.
Popular media reference: look at Jersey shore, which is essentially the high school politics turned up. Many of the actors used very simple strategies, such as Snooki wandering around drunk and saying funny things, or Ronnie essentially just doing plenty of steroids and getting into endless fights.
Other than making sure the robotics hardware looks good, an AI algorithm could be dramatically more compact than the example you gave by developing a “popularity maximizing” policy from the knowledge of many other robots in many other high schools. Most likely, Carl is using a deeply suboptimal policy, not having seen enough training examples in his maximum of 4 years of episodes. (unless he got held back a year). A close to optimal policy, even one with a small compute budget, should greatly outperform Carl.
Lets examine an entirely prosaic situation: Carl, a relatively popular teenager at the local highschool, is deciding whether to invite Bob to this weekend’s party.
some assumptions:
While pondering this decision for an afternoon, Carls’s 10^11 neurons fire 10^2 times per second, for 10^5 seconds, each taking in to account 10^4 input synapses, for 10^22 calculations (extremely roughly)
If there was some route to perform this calculation more efficiently, someone probably would, and would be more popular
The important part of choosing a party invite as the task under consideration, is that I suspect that this is the category of task the human brain is tuned for- and it’s a task that we seem to be naturally inclined to spend enormous amounts of time pondering, alone or in groups- see the trope of the 6 hour pre-prom telephone call. I’m inclined to respect that- to believe that any version of Carl, mechanical or biological, that spent only 10^15 calculations on whether to invite Bob, would eventually get shrecked on the playing field of high school politics.
What model predicts that optimal party planning is as computationally expensive as learning the statistics of the human language well enough to parrot most of human knowledge?
I think your calculations are off by orders of magnitude. Not all neurons fire constantly at 100 times per second—https://aiimpacts.org/rate-of-neuron-firing/ estimates 0.29 to 1.82 times per second. Most importantly perhaps, not all of the processing is directed to that decision. During those hours, many MANY other things are happening.
Thanks for the link to the aiimpacts page! I definitely got the firing rate wrong by about a factor of 50, but I appear to have made other mistakes in the other direction, because I ended up at a number that roughly agrees with aiimpacts- I guessed 10^17 operations per second, and they guess .9 − 33 x 10^16, with low confidence. https://aiimpacts.org/brain-performance-in-flops/
and would be more popular
Not necessarily. In high school politics, pure looks, physical form, and financial support from the parents, all of which are essentially unrelated to brain processing, account for a significant chunk.
Popular media reference: look at Jersey shore, which is essentially the high school politics turned up. Many of the actors used very simple strategies, such as Snooki wandering around drunk and saying funny things, or Ronnie essentially just doing plenty of steroids and getting into endless fights.
Other than making sure the robotics hardware looks good, an AI algorithm could be dramatically more compact than the example you gave by developing a “popularity maximizing” policy from the knowledge of many other robots in many other high schools. Most likely, Carl is using a deeply suboptimal policy, not having seen enough training examples in his maximum of 4 years of episodes. (unless he got held back a year). A close to optimal policy, even one with a small compute budget, should greatly outperform Carl.