StefanHex comments on StefanHex’s Shortform

[StefanHex]

I think this is the sum over the vector dimension, but not over the samples. The sum (mean) over samples is taken later in this line which happens after the division

        metrics[f"{metric_name}"] = torch.cat(metric_values).mean().item()

[Archimedes]

Let’s suppose that’s the case. I’m still not clear on how are you getting to FVU_B?

[StefanHex]

The previous lines calculate the ratio (or 1-ratio) stored in the “explained variance” key for every sample/​batch. Then in that later quoted line, the list is averaged, I.e. we”re taking the sample average over the ratio. That’s the FVU_B formula.

Let me know if this clears it up or if we’re misunderstanding each other!