Regular polygons in models. Recent work studying natural language modular arithmetic has found that language models represent things in a circular fashion. I will contend that “circle” is a bit imprecise; these are actually regular polygons, which are the 2-dimensional versions of polytopes.
A reason why polytopes could be a natural unit of feature geometry is that they characterize linear regions of the activation space in ReLU networks. However, I will note that it’s not clear that this motivation for polytopes coincides very well with the empirical observations above.
[Note] The Polytope Representation Hypothesis
This is an empirical observation about recent works on feature geometry, that (regular) polytopes are a recurring theme in feature geometry.
Simplices in models. Work studying hierarchical structure in feature geometry finds that sets of things are often represented as simplices, which are a specific kind of regular polytope. Simplices are also the structure of belief state geometry.
Regular polygons in models. Recent work studying natural language modular arithmetic has found that language models represent things in a circular fashion. I will contend that “circle” is a bit imprecise; these are actually regular polygons, which are the 2-dimensional versions of polytopes.
A reason why polytopes could be a natural unit of feature geometry is that they characterize linear regions of the activation space in ReLU networks. However, I will note that it’s not clear that this motivation for polytopes coincides very well with the empirical observations above.