With differential geometry, there’s probably a way to translate properties between points. And a way to analyze the geometry of the training distribution: Train the generator to be locally injective and give it an input space uniformly distributed on the unit circle, and whether it successfully trains tells you whether the training distribution has a cycle. Try different input topologies to nail down the distribution’s topology. But just like J’s rank tells you the dimension of the input distribution if you just give the generator enough numbers to work with, a powerful generator ought to tell you the entire topology in one training run...
If the generator’s input distribution is uniform, Σ is diagonal, and the left SVD component of J is also the left (and transposed right) SVD component of JΣJᵀ. Is that useful?
I’d be curious to know whether something like this actually works in practice. It certainly shouldn’t work all the time, since it’s tackling the #P-Hard part of the problem pretty directly, but if it works well in practice that would solve a lot of problems.
With differential geometry, there’s probably a way to translate properties between points. And a way to analyze the geometry of the training distribution: Train the generator to be locally injective and give it an input space uniformly distributed on the unit circle, and whether it successfully trains tells you whether the training distribution has a cycle. Try different input topologies to nail down the distribution’s topology. But just like J’s rank tells you the dimension of the input distribution if you just give the generator enough numbers to work with, a powerful generator ought to tell you the entire topology in one training run...
If the generator’s input distribution is uniform, Σ is diagonal, and the left SVD component of J is also the left (and transposed right) SVD component of JΣJᵀ. Is that useful?
I’d be curious to know whether something like this actually works in practice. It certainly shouldn’t work all the time, since it’s tackling the #P-Hard part of the problem pretty directly, but if it works well in practice that would solve a lot of problems.