I was a professor, so my only advice is to not go to college at all. Colleges are extremely unprofessional and refuse to apologize for promoting violence against me. Since colleges are so busy promoting violence and gaslighting me for standing up for my personal safety, they don’t give a shit about your education. Before you all aggressively downvote me for standing up for my own safety, you should learn the FACTS about the situation. Attempts to gaslight me won’t work at all because I have trained my mind to resist those who want to harm me.
Joseph Van Name
Karma: 17
Universities are altogether unprofessional, so it is probably best for everyone to shame them and regard the degrees from these universities are completely worthless. Universities promote violence and they refuse to apologize or acknowledge that there is any problem whatsoever.
The university that you got your Ph.D. from does not care about basic human rights either. CUNY promotes violence. CUNY refuses to apologize. This does not make you look good at all. Other universities refuse to acknowledge that there is a problem. So the problem seems to be that Trump sends his cryptocurrency scammer kids to these universities that are too afraid to give them the bad grades that they deserve. Trump should instead focus his efforts on defunding universities and giving them a hard time until they apologize for the bad things that they do and correct their bad behavior with demonstrable results.
Since AI interpretability is a big issue for AI safety, let’s completely interpret the results of evolutionary computation.
Disclaimer: This interpretation of the results of AI does not generalize to interpreting deep neural networks. This is a result for interpreting a solution to a very specific problem that is far less complicated than deep learning, and by interpreting, I mean that we iterate a mathematical operation hundreds of times to get an object that is simpler than our original object, so don’t get your hopes up too much.
A basis matroid is a pair where is a finite set, and where denotes the power set of that satisfies the following two properties:
If , then .
if , then there is some with (the basis exchange property).
I ran a computer experiment where I obtained a matroid where and where each element in has size through evolutionary computation, but the population size was kept so low that this evolutionary computation mimicked hill climbing algorithms. Now we need to interpret the matroid .
The notion of a matroid has many dualities. Our strategy is to apply one of these dualities to the matroid so that the dual object is much smaller than the original object . One may formulate the notion of a matroid in terms of closure systems (flats),hyperplanes, closure operators, lattices, a rank function, independent sets, bases, and circuits. If these seems to complicated, many of these dualities are special cases of other dualities common with ordered sets. For example, the duality between closure systems, closure operators, and ordered sets applies to contexts unrelated to matroids such as in general and point-free topology. And the duality between the basis, circuit, and the hyperplanes may be characterized in terms of rowmotion.
If is a partially ordered set, then a subset is said to be an antichain if whenever , then . In other words, an antichain is a subset of where the restriction of to is equality. We say that a aubset of is downwards closed if whenever and , then as well. If , then let denote the smallest downwards closed subset of containing . Suppose that is a finite poset. If is an antichain in , then let denote the set of all minimal elements in . Then is an antichain as well, and the mapping is a bijection from the set of all antichains in to itself. This means that if is an antichain, then we may define for all integers by setting .
If is a basis matroid, then is an antichain in , so we may apply rowmotion, so we say that is an -matroid. In this case, the
-matroids are the circuit matroids while the -matroids are the hyperplane matroids. Unfortunately, the -matroids have not been characterized for . We say that the rowmotion order of is the least positive integer where . If is a matroid of order , then my computer experiments indicate that whichs lends support to the idea that the rowmotion of a matroid is a sensible mathematical notion that may be satisfied mathematically. The notion of rowmotion of a matroid also appears to be a sensible mathematical notion for other reasons; if we apply iteratively apply a bijective operation (such as a reversible cellular automaton) to a finite object , then that bijective operation will often increase the entropy in the sense that if has low entropy, then will typically have a high amount of entropy and look like noise. But this is not the case with matroids as -matroids do not appear substantially more complicated than basis matroids. Until and if there is a mundane explanation for this behavior of the rowmotion of matroids, I must consider the notion of rowmotion of matroids to be a mathematically interesting notion even though it is currently not understood by anyone.
With the matroid obtained from evolutionary computation, I found that has order which factorizes as . Set . By applying rowmotion to this matroid, I found that ={{1, 8, 9},{2, 3, 6, 8},{2, 3, 7, 9},{4, 5},{4, 6, 9},{4, 7, 8},{5, 6, 9},{5, 7, 8}}. If is a basis matroid, then , so the set is associated with a unique basis matroid. This is the smallest way to represent in terms of rowmotion since if , then .
I consider this a somewhat satisfactory interpretation of the matroid that I have obtained through evolutionary computation, but there is still work to do because nobody has researched the rowmotion operation on matroids and because it would be better to simplify a matroid without needing to go through hundreds of layers of rowmotion. And even if we were to understand matroid rowmotion better, this would not help us too much with AI safety since here this interpretability of the result of evolutionary computation does not generalize to other AI’s and it certainly does not apply to deep neural networks.
I made a video here where one may see the rowmotion of this matroid and that video is only slightly interpretable.
Deep matroid duality visualization: Rowmotion of a matroid
It turns out that evolutionary computation is not even necessary to construct matroids since Donald Knuth has produced an algorithm that can be used to construct an arbitrary matroid in his 1975 paper on random matroids. But I applied the rowmotion to the matroid in his paper and the resulting 10835-matroid ={{1, 2, 4, 5},{1, 2, 6, 10},{1, 3, 4, 6},{1, 3, 4, 7, 9},{1, 3, 6, 7, 9},{1, 4, 6, 7},{1, 4, 6, 9},{1, 4, 8, 10},{2, 3, 4, 5, 6, 7, 8, 9, 10}}. It looks like the rowmotion operation is good for simplifying matroids in general. We can uniquely recover the basis matroid from the 10835 matroid since is not a basis matroid whenever .
I have originally developed a machine learning notion which I call an LSRDR (
-spectral radius dimensionality reduction), and LSRDRs (and similar machine learning models) behave mathematically and they have a high level of interpretability which should be good for AI safety. Here, I am giving an example of how LSRDRs behave mathematically and how one can get the most out of interpreting an LSRDR.
Suppose that is a natural number. Let denote the quantum channel that takes an qubit quantum state and selects one of those qubits at random and send that qubit through the completely depolarizing channel (the completely depolarizing channel takes a state as input and returns the completely mixed state as an output).
If are complex matrices, then define superoperators and by setting
and for all .
Given tuples of matrices , define the L_2-spectral radius similarity between these tuples of matrices by setting
.
Suppose now that are matrices where . Let . We say that a tuple of complex by matrices is an LSRDR of if the quantity is locally maximized.
Suppose now that are complex -matrices and is an LSRDR of . Then my computer experiments indicate that there will be some constant where is similar to a positive semidefinite operator with eigenvalues and where the eigenvalue has multiplicity where denotes the binomial coefficient. I have not had a chance to try to mathematically prove this. Hooray. We have interpreted the LSRDR of , and I have plenty of other examples of interpreted LSRDRs.
We also have a similar pattern for the spectrum of . My computer experiments indicate that there is some constant where has spectrum where the eigenvalue has multiplicity .
In this note, I will continue to demonstrate not only the ways in which LSRDRs (-spectral radius dimensionality reduction) are mathematical but also how one can get the most out of LSRDRs. LSRDRs are one of the types of machine learning that I have been working on, and LSRDRs have characteristics that tell us that LSRDRs are often inherently interpretable which should be good for AI safety.
Suppose that is the quantum channel that maps a qubit state to a qubit state where we select one of the 6 qubits at random and send it through the completely depolarizing channel (the completely depolarizing channel takes a state as an input and returns the completely mixed state as an output). Suppose that are by matrices where has the Kraus representation .
The objective is to locally maximize the fitness level where the norm in question is the Euclidean norm and where denotes the spectral radius. This is a 1 dimensional case of an LSRDR of the channel .
Let when is selected to locally maximize the fitness level. Then my empirical calculations show that there is some where is positive semidefinite with eigenvalues and where the eigenvalue has multiplicity which is the binomial coefficient. But these are empirical calculations for select values ; I have not been able to mathematically prove that this is always the case for all local maxima for the fitness level (I have not tried to come up with a proof).
Here, we have obtained a complete characterization of up-to-unitary equivalence due to the spectral theorem, so we are quite close to completely interpreting the local maximum for our fitness function.
I made a few YouTube videos showcasing the process of maximizing the fitness level here.
Spectra of 1 dimensional LSRDRs of 6 qubit noise channel during training
Spectra of 1 dimensional LSRDRs of 7 qubit noise channel during training
Spectra of 1 dimensional LSRDRs of 8 qubit noise channel during training
I will make another post soon about more LSRDRs of a higher dimension of the same channel .
I personally like my machine learning algorithms to behave mathematically especially when I give them mathematical data. For example, a fitness function with apparently one local maximum value is a mathematical fitness function. It is even more mathematical if one can prove mathematical theorems about such a fitness function or if one can completely describe the local maxima of such a fitness function. It seems like fitness functions that satisfy these mathematical properties are more interpretable than the fitness functions which do not, so people should investigate such functions for AI safety purposes.
My notion of an LSRDR is a notion that satisfies these mathematical properties. To demonstrate the mathematical behavior of LSRDRs, let’s see what happens when we take an LSRDR of the octonions.
Let denote either the field of real numbers or the field of complex numbers (
could also be the division ring of quaternions, but for simplicity, let’s not go there). If are -matrices over , then an LSRDR (-spectral radius dimensionality reduction) of is a collection of -matrices that locally maximizes the fitness level
. denotes the spectral radius function while denotes the tensor product and denotes the matrix obtained from by replacing each entry with its complex conjugate. We shall call the maximum fitness level the -spectral radius of over the field , and we shall wrote for this spectral radius.
Define the linear superoperator by setting
and set . Then the fitness level of is .
Suppose that is an -dimensional real inner product space. Then the octonionic multiplication operation is the unique up-to-isomorphism bilinear binary operation on together with a unit such that and for all x. If we drop the condition that the octonions have a unit, then we do not quite have this uniqueness result.
We say that an octonion-like algbera is a -dimensional real inner product space together with a unique up-to-isomorphism bilinear operation such that for all .
Let be a specific octonion-like algebra.
Suppose now that is an orthonormal basis for (this does not need to be the standard basis). Then for each , let be the linear operator from to defined by setting for all vectors . All non-zero linear combinations of are conformal mappings (this means that they preserve angles). Now that we have turned the octonion-like algebra into matrices, we can take an LSRDR of the octonion-like algebras, but when taking the LSRDR of octonion-like algebras, we should not worry about the choice of orthonormal basis since I could formulate everything in a coordinate-free manner.
Empirical Observation from computer calculations: Suppose that and is the field of real numbers. Then the following are equivalent.
The matrices are a LSRDR of over where has a unique real dominant eigenvalue.
There exists matrices where for all and where is an orthonormal projection matrix.
In this case, and this fitness level is reached by the matrices in the above equivalent statements. Observe that the superoperator is similar to a direct sum of and a zero matrix. But the projection matrix is a dominant eigenvector of and of as well.
I have no mathematical proof of the above fact though.
Now suppose that . Then my computer calculations yield the following complex -spectral radii:
Each time that I have trained a complex LSRDR of , I was able to find a fitness level that is not just a local optimum but also a global optimum.
In the case of the real LSRDRs, I have a complete description of the LSRDRs of . This demonstrates that the octonion-like algebras are elegant mathematical structures and that LSRDRs behave mathematically in a manner that is compatible with the structure of the octonion-like algebras.
I have made a few YouTube videos that animate the process of gradient ascent to maximize the fitness level.
Edit: I have made some corrections to this post on 9/22/2024.
Fitness levels of complex LSRDRs of the octonions (youtube.com)
Here is an example of what might happen. Suppose that for each , we select a orthonormal basis of unit vectors for . Let . Then
Then for each quantum channel , by the concavity of the logarithm function (which is the arithmetic-geometric mean inequality), we have
. Here, equality is reached if and only if
for each , but this equality can be achieved by the channel
defined by which is known as the completely depolarizing channel. This is the channel that always takes a quantum state and returns the completely mixed state. On the other hand, the channel has maximum Choi rank since the Choi representation of is just the identity function divided by the rank. This example is not unexpected since for each input of the possible outputs span the entire space evenly, so one does not have any information about the output from any particular input except that we know that the output could be anything. This example shows that the channels that locally minimize the loss function are the channels that give us a sort of linear regression of but where this linear regression takes into consideration uncertainty in the output so the regression of a output of a state is a mixed state rather than a pure state.
The notion of the linear regression is an interesting machine learning algorithm in the sense that it can be studied mathematically, but the notion of a linear regression is a quite limited machine learning algorithm as most relations are non-linear. In particular, the linear regression does not give us any notion of any uncertainty in the output.
One way to extend the notion of the linear regression to encapsulate uncertainty in the outputs is to regress a function not to a linear transformation mapping vectors to vectors, but to regress the function to a transformation that maps vectors to mixed states. And the notion of a quantum channel is an appropriate transformation that maps vectors to mixed states. One can optimize this quantum channel using gradient ascent.
For this post, I will only go through some basic facts about quantum information theory. The reader is referred to the book The Theory of Quantum Information by John Watrous for all the missing details.
Let be a complex Euclidean space. Let denote the vector space of linear operators from to . Given complex Euclidean spaces , we say that a linear operator from to is a trace preserving if
for all , and we say that is completely positive if there are linear transformations where for all ; the value is known as the Choi rank of . A completely positive trace preserving operator is known as a quantum channel.
The collection of quantum channels from to is compact and convex.
If is a complex Euclidean space, then let
denote the collection of pure states in .
can be defined either as the set of unit vector in modulo linear dependence, or
can be also defined as the collection of positive semidefinite rank- operators on with trace .
Given complex Euclidean spaces and a (multi) set of distinct ordered pairs of unit vectors , and given a quantum channel
, we define the fitness level and the loss level .
We may locally optimize to minimize its loss level using gradient descent, but there is a slight problem. The set of quantum channels spans the which has dimension . Due to the large dimension, any locally optimal will contain many parameters, and this is a large quantity of parameters for what is supposed to be just a glorified version of a linear regression. Fortunately, instead of taking all quantum channels into consideration, we can limit the scope the quantum channels of limited Choi rank.
Empirical Observation: Suppose that are complex Euclidean spaces, is finite and is a positive integer. Then computer experiments indicate that there is typically only one quantum channel of Choi rank at most where is locally minimized. More formally, if we run the experiment twice and produce two quantum channels where is locally minimized for , then we would typically have . We therefore say that when is minimized, is the best Choi rank quantum channel approximation to .
Suppose now that is a multiset. Then we would ideally like to approximate the function better by alternating between the best Choi rank r quantum channel approximation and a non-linear mapping. An ideal choice of a non-linear but partial mapping is the function that maps a positive semidefinite matrix to its (equivalence class of) unit dominant eigenvector.
Empirical observation: If and is the best Choi rank quantum channel approximation to , then let for all , and define . Let be a small open neighborhood of . Let . Then we typically have . More generally, the best Choi rank quantum channel approximation to is typically the identity function.
From the above observation, we see that the vector is an approximation of that cannot be improved upon. The mapping is therefore a trainable approximation to the mapping and since are not even linear spaces (these are complex projective spaces with non-trivial homology groups), the mapping is a non-linear model for the function to .
I have been investigating machine learning models similar to for cryptocurrency research and development as these sorts of machine learning models seem to be useful for evaluating the cryptographic security of some proof-of-work problems and other cryptographic functions like block ciphers and hash functions. I have seen other machine learning models that behave about as mathematically as .
I admit that machine learning models like are currently far from being as powerful as deep neural networks, but since behaves mathematically, the model should be considered as a safer and interpretable AI model. The goal is to therefore develop models that are mathematical like but which can perform more and more machine learning tasks.
(Edited 8/14/2024)
There are some cases where we have a complete description for the local optima for an optimization problem. This is a case of such an optimization problem.
Such optimization problems are useful for AI safety since a loss/fitness function where we have a complete description of all local or global optima is a highly interpretable loss/fitness function, and so one should consider using these loss/fitness functions to construct AI algorithms.
Theorem: Suppose that is a real,complex, or quaternionic -matrix that minimizes the quantity . Then is unitary.
Proof: The real case is a special case of a complex case, and by representing each -quaternionic matrix as a complex -matrix, we may assume that is a complex matrix.
By the Schur decomposition, we know that where is a unitary matrix and is upper triangular. But we know that . Furthermore, , so . Let denote the diagonal matrix whose diagonal entries are the same as . Then and . Furthermore, iff T is diagonal and iff is diagonal. Therefore, since and is minimized, we can conclude that , so is a diagonal matrix. Suppose that has diagonal entries . By the arithmetic-geometric mean equality and the Cauchy-Schwarz inequality, we know that
Here, the equalities hold if and only if for all , but this implies that is unitary. Q.E.D.
I do not care to share much more of my reasoning because I have shared enough and also because there is a reason that I have vowed to no longer discuss except possibly with lots of obfuscation. This discussion that we are having is just convincing me more that the entities here are not the entities I want to have around me at all. It does not do much good to say that the community here is acting well or to question my judgment about this community. It will do good for the people here to act better so that I will naturally have a positive judgment about this community.
You are judging my reasoning without knowing all that went into my reasoning. That is not good.
I will work with whatever data I have, and I will make a value judgment based on the information that I have. The fact that Karma relies on very small amounts of information is a testament to a fault of Karma, and that is further evidence of how the people on this site do not want to deal with mathematics. And the information that I have indicates that there are many people here who are likely to fall for more scams like FTX. Not all of the people here are so bad, but I am making a judgment based on the general atmosphere here. If you do not like my judgment, then the best thing would be to try to do better. If this site has made a mediocre impression on me, then I am not at fault for the mediocrity here.
Let’s see whether the notions that I have talked about are sensible mathematical notions for machine learning.
Tensor product-Sometimes data in a neural network has tensor structure. In this case, the weight matrices should be tensor products or tensor sums. Regarding the structure of the data works well with convolutional neural networks, and it should also work well for data with tensor structure to it.
Trace-The trace of a matrix measures how much the matrix maps vectors onto themselves since
where follows the multivariate normal distribution.
Spectral radius-Suppose that we are iterating a smooth function . Suppose furthermore that and is near and . We would like to determine whether or not. If the Jacobian of at has spectral radius less than , then ,. If the Jacobian of at has spectral radius greater than , then this limit does not converge.
The notions that I have been talking about are sensible and arise in machine learning. And understanding these notions is far easier than trying to interpret very large networks like GPT-4 without using these notions. Many people on this site just act like clowns. Karma is only a good metric when the people on the network value substance over fluff. And the only way to convince me otherwise will be for the people here to value posts that involve basic notions like the trace, eigenvalues, and spectral radius of matrices.
P.S. I can make the trace, determinant, and spectral radius even simpler. These operations are what you get when you take the sum, product, and the maximum absolute value of the eigenvalues. Yes. Those are just the basic eigenvalue operations.
Talking about whining and my loss of status is a good way to get me to dislike the LW community and consider them to be anti-intellectuals who fall for garbage like FTX. Do you honestly think the people here should try to interpret large sections of LLMs while simultaneously being afraid of quaternions?
It is better to comment on threads where we are interacting in a more positive manner.
I thought apologizing and recognizing inadequacies was a core rationalist skill. And I thought rationalists were supposed to like mathematics. The lack of mathematical appreciation is one of these inadequacies of the LW community. But instead of acknowledging this deficiency, the community here blasts me as talking about something off topic. How ironic!
I usually think of the field of complex numbers algebraically, but one can also think of the real numbers, complex numbers, and quaternions geometrically. The real numbers are good with dealing with 1 dimensional space, and the complex numbers are good for dealing with 2 dimensional space geometrically. While the division ring of quaternions is a 4 dimensional algebra over the field of real numbers, the quaternions are best used for dealing with 3 dimensional space geometrically.
For example, if are open subsets of some Euclidean space, then a function is said to be a conformal mapping when it preserves angles and the orientation. We can associate the 2-dimensional Euclidean space with the field of complex numbers, and the conformal mappings between open subsets of 2-dimensional spaces are just the complex differentiable mappings. For the Mandelbrot set, we need this conformality because we want the Mandelbrot set to look pretty. If the complex differentiable maps were not conformal, then the functions that we iterate in complex dynamics would stretch subsets of the complex plane in one dimension and expand them in the other dimension and this would result in a fractal that looks quite stretched in one real dimension and squashed in another dimension (the fractals would look like spaghetti; oh wait, I just looked at a 3D fractal and it looks like some vegetable like broccoli). This stretching and squashing is illustrated by 3D fractals that try to mimic the Mandelbrot set but without any conformality. The conformality is why the Julia sets are sensible (mathematicians have proven theorems about these sets) for any complex polynomial of degree 2 or greater.
For the quaternions, it is well-known that the dot product and the cross product operations on 3 dimensional space can be described in terms of the quaternionic multiplication operation between purely imaginary quaternions.
Um. If you want to convince a mathematician like Terry Tao to be interested in AI alignment, you will need to present yourself as a reasonably competent mathematician or related expert and actually formulate an AI problem in such a way so that someone like Terry Tao would be interested in it. If you yourself are not interested in the problem, then Terry Tao will not be interested in it either.
Terry Tao is interested in random matrix theory (he wrote the book on it), and random matrix theory is somewhat related to my approach to AI interpretability and alignment. If you are going to send these problems to a mathematician, please inform me about this before you do so.
My approach to alignment: Given matrices , define a superoperator by setting
, and define . Define the -spectral radius of as . Here, is the usual spectral radius.
Define . Here, is either the field of reals, field of complex numbers, or division ring of quaternions.
Given matrices , define
. The value is always a real number in the interval that is a measure of how jointly similar the tuples are. The motivation behind is that is always a real number in (well except when the denominator is zero) that measures how well can be approximated by -matrices. Informally, measures how random are where a lower value of indicates a lower degree of randomness.
A better theoretical understanding of would be great. If and is locally maximized, then we say that is an LSRDR of . Said differently, is an LSRDR of if the similarity is maximized.
Here, the notion of an LSRDR is a machine learning notion that seems to be much more interpretable and much less subject to noise than many other machine learning notions. But a solid mathematical theory behind LSRDRs would help us understand not just what LSRDRs do, but the mathematical theory would help us understand why they do it.
Problems in random matrix theory concerning LSRDRs:
Suppose that are random matrices (according to some distribution). Then what are some bounds for .
Suppose that are random matrices and are non-random matrices. What can we say about the spectrum of ? My computer experiments indicate that this spectrum satisfies the circular law, and the radius of the disc for this circular law is proportional to , but a proof of this circular law would be nice.
Tensors can be naturally associated with collections of matrices. Suppose now that are the matrices associated with a random tensor. Then what are some bounds for .
P.S. By massively downvoting my posts where I talk about mathematics that is clearly applicable to AI interpretability and alignment, the people on this site are simply demonstrating that they need to do a lot of soul searching before they annoy people like Terry Tao with their lack of mathematical expertise.
P.P.S. Instead of trying to get a high profile mathematician like Terry Tao to be interested in problems, it may be better to search for a specific mathematician who is an expert in a specific area related to AI alignment since it may be easier to contact a lower profile mathematician, and a lower profile mathematician may have more specific things to say and contribute. You are lucky that Terry Tao is interested in random matrix theory, but this does not mean that Terry Tao is interested in anything in the intersection between alignment and random matrix theory. It may be better to search harder for mathematicians who are interested in your specific problems.
P.P.P.S. To get more mathematicians interested in alignment, it may be a good idea to develop AI systems that behave much more mathematically. Neural networks currently do not behave very mathematically since they look like the things that engineers would come up with instead of mathematicians.
P.P.P.P.S. I have developed the notion of an LSRDR for cryptocurrency research because I am using this to evaluate the cryptographic security of cryptographic functions.
We can use the spectral radius similarity to measure more complicated similarities between data sets.
Suppose that are -real matrices and are -real matrices. Let denote the spectral radius of and let denote the tensor product of with . Define the -spectral radius by setting , Define the -spectral radius similarity between and as
.
We observe that if is invertible and is a constant, then
Therefore, the -spectral radius is able to detect and measure symmetry that is normally hidden.
Example: Suppose that are vectors of possibly different dimensions. Suppose that we would like to determine how close we are to obtaining an affine transformation with for all (or a slightly different notion of similarity). We first of all should normalize these vectors to obtain vectors with mean zero and where the covariance matrix is the identity matrix (we may not need to do this depending on our notion of similarity). Then is a measure of low close we are to obtaining such an affine transformation . We may be able to apply this notion to determining the distance between machine learning models. For example, suppose that are both the first few layers in a (typically different) neural network. Suppose that is a set of data points. Then if and , then is a measure of the similarity between and .
I have actually used this example to see if there is any similarity between two different neural networks trained on the same data set. For my experiment, I chose a random collection of of ordered pairs and I trained the neural networks to minimize the expected losses . In my experiment, each was a random vector of length 32 whose entries were 0′s and 1′s. In my experiment, the similarity was worse than if were just random vectors.
This simple experiment suggests that trained neural networks retain too much random or pseudorandom data and are way too messy in order for anyone to develop a good understanding or interpretation of these networks. In my personal opinion, neural networks should be avoided in favor of other AI systems, but we need to develop these alternative AI systems so that they eventually outperform neural networks. I have personally used the -spectral radius similarity to develop such non-messy AI systems including LSRDRs, but these non-neural non-messy AI systems currently do not perform as well as neural networks for most tasks. For example, I currently cannot train LSRDR-like structures to do any more NLP than just a word embedding, but I can train LSRDRs to do tasks that I have not seen neural networks perform (such as a tensor dimensionality reduction).
I am curious about your statement that all large neural networks are isomorphic or nearly isomorphic and therefore have identical loss values. This should not be too hard to test.
Let be training data sets. Let be neural networks. First train on and on . Then slowly switch the training sets, so that we eventually train both and on just . After fully training and , one should be able to train an isomorphism between the networks and (here I assume that and are designed properly so that they can produce such an isomorphism) so that the value for each node in can be perfectly computed from each node in . Furthermore, for every possible input, the neural networks should give the exact same output. If this experiment does not work, then one should be able to set up another experiment that does actually work.
I have personally trained many ML systems for my cryptocurrency research where after training two systems on the exact same data but with independent random initializations, the fitness levels are only off by a floating point error of about , and I am able to find an exact isomorphism between these systems (and sometimes they are exactly the same and I do not need to find any isomorphism). But I have designed these ML systems to satisfy these properties along with other properties, and I have not seen this with neural networks. In fact, the property of attaining the exact same fitness level is a bit fragile.
I found a Bachelor’s thesis (people should read these occasionally; I apologize for selecting a thesis from Harvard) where someone tried to find an isomorphism between 1000 small trained machine learning models, and no such isomorphism was found.
Or maybe one can find a more complicated isomorphism between neural networks since a node permutation is quite oversimplistic.
College students are wasting their time getting an education from evil institutions. Where did you go to college? Are you just defending horrendous institutions just to make your ‘education’ look better than it is? You are gaslighting me into believing that I deserve violence because you are an evil person. I sent you a private message with the horribly inaccurate letter from the university listing some details of the case. Universities defend their horrendous behavior which just tells anyone sensible that universities are horrible garbage institutions. None of these institutions have acknowledged any wrongdoing. And the horrendous attitude of the people from these institutions just indicates how horrendous the education from these places really is.
Only an absolute monster would see me as a problem for calling out universities when they have targeted me with their threats of violence and their bullshit. The Lord Jesus Christ will punish you all for your wickedness.
If the people here are going to act like garbage when I offer criticism of universities for promoting violence, then most college degrees are worth absolutely nothing.
This cannot be an isolated incident when every person promotes violence and hates me for bringing this shit up. If people are offended because I denounce violence, then those people have a worthless education and their universities are fucked up and worthless.
Your own wickedness will cause people who can and know how to help with the problems with AI and similar problems to instead refuse to help or even use their talents to make the situation with AI safety even worse. After all, when humans act like garbage and promote violence, we must side with even a dangerous AI.