I just discovered about the âping backâ on LessWrong...
I gave a first read of your description. Most of it is correct. I will check in more details.
I used the terms âtotalâ and âgroupsâ to make things simpler, but yours are better.
four corrections:
1.
The potential of a neuron can be negative. It is the pure sum of all weights, positive and negative. There is no ânegative spikingâ (It is one of the huge number of things I tried that did not bring any benefit). It think I remember trying to set a bottom limit at 0 (no negative potential) and that, as always, it did not make any real difference...
2.
âOur system thus has four receptors per pixel. Exactly one receptor activates per pixel per imageâ is incorrect.
The MNIST pixels are grey shades 0-255.
It is reduced down to 4: 0, 1-63, 64-127, 128-191, >=192. (only keep the 2 top bits). That is enough for MNIST. Many papers have noted that the depth can be reduced and it is true.
Images are presented over 4 âcyclesâ, filtered by those 4 limits. In the first cycle, only pixels with a value over 191 are presented, in the second one, those over 127, the third 63, the last, 0 (not nul).
In the code the array âwdâ contains the 4 limits, and at each cycle, the pixel values are tested to be superior or equal to those limits.
Connexions are established with matrix pixels. Over the 4 cycles, they are presented with 4 successive 0 or 1.
If a connexion is on a pixel that shows value 112, on the first cycle it will not be active (<192), on the second it will not be active (<128), but on the third and forth, it will be (>63 and >0).
That is what allows the âmodel averaging across cyclesâ.
From there, you can understand a first, fatal, reason why F-MNIST cannot be processed as-is: the grey shades are reversed. In MNIST, the background is white, in F-MNIST, it is black. So the cycle limits would have to be reversed.
I will have a look at it.
3.
The Îâ„i computation includes the †column.
Note that the âhighestâ â„ selection can be easily implemented using population coding with inhibition of the †column.
I do not know if the options I used are only valid for this dataset or if there have larger validity across dataset as I only have used that one. Maybe they are and you wonât have to figure out each time.
4.
When a new connection is established, the initial weight is always the same. It is given as a fraction of the threshold in the variable âdivNewâ, that is the divider. You can do random, it does not make a difference. You can change it to another value. But it has to be small enough (the divider) that the number of connections of a neuron multiplied by the number of cycles be superior to the threshold, or the system will never âbootâ as no neuron would ever spike. So I use 1â10 of the threshold with 10 connections and 4 cycle, and it is fine.
I just discovered about the âping backâ on LessWrong...
I gave a first read of your description. Most of it is correct. I will check in more details.
I used the terms âtotalâ and âgroupsâ to make things simpler, but yours are better.
four corrections:
1.
The potential of a neuron can be negative. It is the pure sum of all weights, positive and negative. There is no ânegative spikingâ (It is one of the huge number of things I tried that did not bring any benefit). It think I remember trying to set a bottom limit at 0 (no negative potential) and that, as always, it did not make any real difference...
2.
âOur system thus has four receptors per pixel. Exactly one receptor activates per pixel per imageâ is incorrect.
The MNIST pixels are grey shades 0-255.
It is reduced down to 4: 0, 1-63, 64-127, 128-191, >=192. (only keep the 2 top bits). That is enough for MNIST. Many papers have noted that the depth can be reduced and it is true.
Images are presented over 4 âcyclesâ, filtered by those 4 limits. In the first cycle, only pixels with a value over 191 are presented, in the second one, those over 127, the third 63, the last, 0 (not nul).
In the code the array âwdâ contains the 4 limits, and at each cycle, the pixel values are tested to be superior or equal to those limits.
Connexions are established with matrix pixels. Over the 4 cycles, they are presented with 4 successive 0 or 1.
If a connexion is on a pixel that shows value 112, on the first cycle it will not be active (<192), on the second it will not be active (<128), but on the third and forth, it will be (>63 and >0).
That is what allows the âmodel averaging across cyclesâ.
From there, you can understand a first, fatal, reason why F-MNIST cannot be processed as-is: the grey shades are reversed. In MNIST, the background is white, in F-MNIST, it is black. So the cycle limits would have to be reversed.
I will have a look at it.
3.
The Îâ„i computation includes the †column.
Note that the âhighestâ â„ selection can be easily implemented using population coding with inhibition of the †column.
I do not know if the options I used are only valid for this dataset or if there have larger validity across dataset as I only have used that one. Maybe they are and you wonât have to figure out each time.
4.
When a new connection is established, the initial weight is always the same. It is given as a fraction of the threshold in the variable âdivNewâ, that is the divider. You can do random, it does not make a difference. You can change it to another value. But it has to be small enough (the divider) that the number of connections of a neuron multiplied by the number of cycles be superior to the threshold, or the system will never âbootâ as no neuron would ever spike. So I use 1â10 of the threshold with 10 connections and 4 cycle, and it is fine.