In practice, this required looking at altogether thousands of panels of interactive PCA plots like this [..]
Most clusters however don’t seem obviously interesting.
What do you think of @jake_mendel’s point about the streetlight effect?
If the methodology was looking at 2D slices of up to a 5 dimensional spaces, was detection of multi-dimensional shapes necessarily biased towards human identification and signaling of shape detection in 2D slices?
I really like your update to the superposition hypothesis from linear to multi-dimensional in your section 3, but I’ve been having a growing suspicion that—especially if node multi-functionality and superposition is the case—that the dimensionality of the data compression may be severely underestimated. If Llama on paper is 4,096 dimensions, but in actuality those nodes are superimposed, there could be OOM higher dimensional spaces (and structures in those spaces) than the on paper dimensionality max.
So even if your revised version of the hypothesis is correct, it might be that the search space for meaningful structures was bounded much lower than where the relatively ‘low’ composable mulit-dimensional shapes are actually primarily forming.
I know that for myself, even when considering basic 4D geometry like a tesseract, if data clusters were around corners of the shape I’d only spot a small number of the possible 2D slices, and in at least one of those cases might think what I was looking at was a circle instead of a tesseract: https://mathworld.wolfram.com/images/eps-gif/TesseractGraph_800.gif
Do you think future work may be able to rely on automated multi-dimensional shape and cluster detection exploring shapes and dimensional spaces well beyond even just 4D, or that the difficulty in mutli-dimensional pattern recognition will remain a foundational obstacle for the foreseeable future?
What do you think of @jake_mendel’s point about the streetlight effect?
If the methodology was looking at 2D slices of up to a 5 dimensional spaces, was detection of multi-dimensional shapes necessarily biased towards human identification and signaling of shape detection in 2D slices?
I really like your update to the superposition hypothesis from linear to multi-dimensional in your section 3, but I’ve been having a growing suspicion that—especially if node multi-functionality and superposition is the case—that the dimensionality of the data compression may be severely underestimated. If Llama on paper is 4,096 dimensions, but in actuality those nodes are superimposed, there could be OOM higher dimensional spaces (and structures in those spaces) than the on paper dimensionality max.
So even if your revised version of the hypothesis is correct, it might be that the search space for meaningful structures was bounded much lower than where the relatively ‘low’ composable mulit-dimensional shapes are actually primarily forming.
I know that for myself, even when considering basic 4D geometry like a tesseract, if data clusters were around corners of the shape I’d only spot a small number of the possible 2D slices, and in at least one of those cases might think what I was looking at was a circle instead of a tesseract: https://mathworld.wolfram.com/images/eps-gif/TesseractGraph_800.gif
Do you think future work may be able to rely on automated multi-dimensional shape and cluster detection exploring shapes and dimensional spaces well beyond even just 4D, or that the difficulty in mutli-dimensional pattern recognition will remain a foundational obstacle for the foreseeable future?