I really like this article. Just a bit of brainstorming, something that has almost certainly been tried:
a. You can feed in all your examples of a specific class into these image detectors and get a list of which neurons have non-negligible activations
b. You can then break a trained and effective [multi-class image detector] into separate [single class detectors] from just the neurons that ‘matter’
c. Some of these detectors may have ‘evolved’ a better way to detect their class than others. You can then do a variety of mixing and matching by architecture transfer from a ‘good’ detector to a weak detector.
Essentially my thought is that while it may be exhaustive in human effort to really ‘understand’ how a given subsystem is done, you should be able to get closer to true global maxima in performance by taking the best examples of a repeating element that gets rediscovered over and over, and basically plugging it in everywhere this element will help.
It seems like there are many different instances of ‘duplicated function’ in networks, and like duplicated code, some of the code is going to be better than others.
I really like this article. Just a bit of brainstorming, something that has almost certainly been tried:
a. You can feed in all your examples of a specific class into these image detectors and get a list of which neurons have non-negligible activations
b. You can then break a trained and effective [multi-class image detector] into separate [single class detectors] from just the neurons that ‘matter’
c. Some of these detectors may have ‘evolved’ a better way to detect their class than others. You can then do a variety of mixing and matching by architecture transfer from a ‘good’ detector to a weak detector.
Essentially my thought is that while it may be exhaustive in human effort to really ‘understand’ how a given subsystem is done, you should be able to get closer to true global maxima in performance by taking the best examples of a repeating element that gets rediscovered over and over, and basically plugging it in everywhere this element will help.
It seems like there are many different instances of ‘duplicated function’ in networks, and like duplicated code, some of the code is going to be better than others.
Moreover, these methods would be general-purpose.