emiddell comments on Mapping the semantic void: Strange goings-on in GPT embedding spaces

emiddell 15 Dec 2023 10:06 UTC
4 points
1
In higher dimensions most of the volume of a n-sphere is found close to its shell.
The volume of such a sphere is $V_{n} (R) \propto R^{n}$ . [1]
The ratio of the volume of a shell to the rest of the ball is
$\frac{V_{n} (R + Δ) - V_{n} (R)}{V_{n} (R)} = (1 + \frac{Δ}{R})^{n} - 1$
which grows quickly with n.
The embedding places the tokens in areas where there is the most space to accommodate them.
[1] https://en.wikipedia.org/wiki/Volume_of_an_n-ball
- jacobjacob 15 Dec 2023 18:57 UTC
  4 points
  0
  Parent
  Heads up, we support latex :)
  Use Ctrl-4 to open the LaTex prompt (or Cmd-4 if you’re on a Mac). Open a centred LaTex popup using Ctrl-M (aka Cmd-M). If you’ve written some maths in normal writing and want to turn it into LaTex, if you highlight the text and then hit the LaTex editor button it will turn straight into LaTex.
  https://www.lesswrong.com/posts/xWrihbjp2a46KBTDe/editor-mini-guide