Update: I found a proof of the “exponential number of near-orthogonal vectors” in these lecture notes https://www.cs.princeton.edu/courses/archive/fall16/cos521/Lectures/lec9.pdf
From my understanding, the proof uses a quantification of just how likely near-orthogonality becomes in high-dimensional spaces and derives a probability for pairwise near-orthogonality of many states.
This does not quite help my intuitions, but I’ll just assume that the “it it possible to tile the surface efficiently with circles even if their size gets close to the 45° threshold” resolves to “yes, if the dimensionality is high enough”.
One interesting aspect of these considerations should be that with growing dimensionality the definition of near-orthogonality can be made tighter without loosing the exponential number of vectors. This should define a natural signal-to-noise ratio for information encoded in this fashion.
Update: I found a proof of the “exponential number of near-orthogonal vectors” in these lecture notes https://www.cs.princeton.edu/courses/archive/fall16/cos521/Lectures/lec9.pdf From my understanding, the proof uses a quantification of just how likely near-orthogonality becomes in high-dimensional spaces and derives a probability for pairwise near-orthogonality of many states.
This does not quite help my intuitions, but I’ll just assume that the “it it possible to tile the surface efficiently with circles even if their size gets close to the 45° threshold” resolves to “yes, if the dimensionality is high enough”.
One interesting aspect of these considerations should be that with growing dimensionality the definition of near-orthogonality can be made tighter without loosing the exponential number of vectors. This should define a natural signal-to-noise ratio for information encoded in this fashion.