Hey, i’m controlling the sparsity when I’m creating the batch of the data, so during that time, i sample according to the probability i’m assigning for that feature.
re: features getting baked into the bias: yeah, that might be one of the intuitions we can develop but to me the interesting part is that that kind of behaviour didn’t happen in any of the other cases when the importance was varying and just happened when the feature importance for all of them is equal. I don’t have a concrete intuition on why that might be the case, still trying to think on it.
Hey, i’m controlling the sparsity when I’m creating the batch of the data, so during that time, i sample according to the probability i’m assigning for that feature.
re: features getting baked into the bias: yeah, that might be one of the intuitions we can develop but to me the interesting part is that that kind of behaviour didn’t happen in any of the other cases when the importance was varying and just happened when the feature importance for all of them is equal. I don’t have a concrete intuition on why that might be the case, still trying to think on it.