Neat, thanks. Later I might want to rederive the estimates using different assumptions—not only should the number of active features L be used in calculating average ‘noise’ level (basically treating it as an environment parameter rather than a design decision), but we might want another free parameter for how statistically dependent features are. If I really feel energetic I might try to treat the per-layer information loss all at once rather than bounding it above as the sum of information losses of individual features.
Neat, thanks. Later I might want to rederive the estimates using different assumptions—not only should the number of active features L be used in calculating average ‘noise’ level (basically treating it as an environment parameter rather than a design decision), but we might want another free parameter for how statistically dependent features are. If I really feel energetic I might try to treat the per-layer information loss all at once rather than bounding it above as the sum of information losses of individual features.