That’s because a classifier only needs to find a direction which correctly classifies the data, not a direction which makes other classifiers fail. A direction which removes all linearly available information is not always as good as the direction found with LR (at classification).
Maybe figure 2 from the paper which introduced mean difference as a way to remove linear information might help.
That’s because a classifier only needs to find a direction which correctly classifies the data, not a direction which makes other classifiers fail. A direction which removes all linearly available information is not always as good as the direction found with LR (at classification).
Maybe figure 2 from the paper which introduced mean difference as a way to remove linear information might help.