in the space of binary-sequences of all lengths, i have an intuition that {the rate at which there are new ‘noticed patterns’ found at longer lengths} decelerates as the length increases.
what do i mean by “noticed patterns”?
in some sense of ‘pattern’, each full sequence is itself a ‘unique pattern’. i’m using this phrase to avoid that sense.
rather, my intuition is that {what could in principle be noticed about sequences of higher lengths} exponentially tends to be things that had already been noticed of sequences of lower lengths. ‘meta patterns’ and maybe ‘classes’ are other possible terms for these. two simple examples are “these ones are all random-looking sequences” and “these can be compressed in a basic way”[1].
note: not implying there are few such “meta-patterns that can be noticed about a sequence”, or that most would be so simple/human-comprehensible.
in my intuition this generalizes to functions/programs in general. as an example: in the space of all definable ‘mathematical universes’, ‘contains agentic processes’ is such a meta-pattern which would continue to recur (=/= always or usually present) at higher description lengths.
(‘mathematical universe’ does not feel like a distinctly-bounded category to me. i really mean ‘very-big/complex programs’, and ‘universe’ can be replaced with ‘program’. i just use this phrasing to try to help make this understandable, because i expect the claim that ‘contains agents’ is such a recurring higher-level pattern to be intuitive.)
and as you consider universes/programs whose descriptions are increasingly complex, eventually ~nothing novel could be noticed. e.g., you keep seeing worlds where agentic processes are dominant, or where some simple unintelligent process cascades into a stable end equilibrium, or where there’s no potential for those, etc <same note from earlier applies>. (more-studied things like computational complexity may also be examples of such meta-patterns)
a stronger claim which might follow (about the space of possible programs) is that eventually (at very high lengths), even as length/complexity increases exponentially, the resulting universes/programs higher-level behavior[2] still ends up nearly-isomorphic to that of relatively-much-earlier/simpler universes/programs. (incidentally, this could be used to justify a simplicity prior/heuristic)
in conclusion, if this intuition is true, the space of all functions/programs is ‘already’ or naturally a space of constrained diversity. in other words, if true, the space of meta-patterns[3] is finite (i.e approaches some specific integer), even though the space of functions/programs is infinite.
though this makes me wonder about the possibility of ‘anti-pattern’ programs i.e ones selected/designed to not be nearly-isomorphic to anything previous. maybe they’d become increasingly sparse or something?
for some given formal definition that matches what the ‘meta/noticed pattern’ concept is trying to be about, which i don’t know how to define. this concept also does not feel distinctly-bounded to me, so i guess there’s multiple corresponding definitions
Consider all the programs P that encode uncomputable numbers up to n digits. There are infinitely many of these programs. Now consider the set of programs P′:={call-10-times(p)|p∈P}. Each program in P’ has some pattern. But it’s always a different one.
in the space of binary-sequences of all lengths, i have an intuition that {the rate at which there are new ‘noticed patterns’ found at longer lengths} decelerates as the length increases.
what do i mean by “noticed patterns”?
in some sense of ‘pattern’, each full sequence is itself a ‘unique pattern’. i’m using this phrase to avoid that sense.
rather, my intuition is that {what could in principle be noticed about sequences of higher lengths} exponentially tends to be things that had already been noticed of sequences of lower lengths. ‘meta patterns’ and maybe ‘classes’ are other possible terms for these. two simple examples are “these ones are all random-looking sequences” and “these can be compressed in a basic way”[1].
note: not implying there are few such “meta-patterns that can be noticed about a sequence”, or that most would be so simple/human-comprehensible.
in my intuition this generalizes to functions/programs in general. as an example: in the space of all definable ‘mathematical universes’, ‘contains agentic processes’ is such a meta-pattern which would continue to recur (=/= always or usually present) at higher description lengths.
(‘mathematical universe’ does not feel like a distinctly-bounded category to me. i really mean ‘very-big/complex programs’, and ‘universe’ can be replaced with ‘program’. i just use this phrasing to try to help make this understandable, because i expect the claim that ‘contains agents’ is such a recurring higher-level pattern to be intuitive.)
and as you consider universes/programs whose descriptions are increasingly complex, eventually ~nothing novel could be noticed. e.g., you keep seeing worlds where agentic processes are dominant, or where some simple unintelligent process cascades into a stable end equilibrium, or where there’s no potential for those, etc <same note from earlier applies>. (more-studied things like computational complexity may also be examples of such meta-patterns)
a stronger claim which might follow (about the space of possible programs) is that eventually (at very high lengths), even as length/complexity increases exponentially, the resulting universes/programs higher-level behavior[2] still ends up nearly-isomorphic to that of relatively-much-earlier/simpler universes/programs. (incidentally, this could be used to justify a simplicity prior/heuristic)
in conclusion, if this intuition is true, the space of all functions/programs is ‘already’ or naturally a space of constrained diversity. in other words, if true, the space of meta-patterns[3] is finite (i.e approaches some specific integer), even though the space of functions/programs is infinite.
(e.g., 100
1s followed by 1000s is simple to compress)though this makes me wonder about the possibility of ‘anti-pattern’ programs i.e ones selected/designed to not be nearly-isomorphic to anything previous. maybe they’d become increasingly sparse or something?
for some given formal definition that matches what the ‘meta/noticed pattern’ concept is trying to be about, which i don’t know how to define. this concept also does not feel distinctly-bounded to me, so i guess there’s multiple corresponding definitions
Consider all the programs P that encode uncomputable numbers up to n digits. There are infinitely many of these programs. Now consider the set of programs P′:={call-10-times(p)|p∈P}. Each program in P’ has some pattern. But it’s always a different one.