Task: convert mathematical expressions into natural language
Context: A researcher is reading a paper about alignment that contains a lot of well-specified but dense mathematical notation. They would like to see a less terse and more fluent description of the same idea that’s easier to read, similar to what a researcher might say to them at a blackboard while writing the math. This might involve additional context for novices.
Input type: a piece of mathematically-dense but well-specified text from a paper
Output type: a fluent, natural language descirption of the same mathematical objects
Output: Consider a predictive model that predicts the output of a video camera given some sensors in the world. As in ELK, our goal will be to find a function that looks at the sensors, the model, and some questions, then reutrns some answers to those questions using the model’s latent knoweldge.
Output: Alice has n things that she values: given any of these items, she’ll always value a set at least as much if she adds another one. A robot is given a proxy for this utility, but it dpeends on a strict subset of the items. The robot optimizes its proxy subjec to some resource constrants. It’s a thoerem that the robot will not pick the things that weren’t included in its utility.
Output: We think of events, in the sense of the sigma algebras that are used in the formalization fo probability theory. Each event has a probability and an expected utility assigned to it. We are interested in the product of these two, which Valdimir Nesov called “shouldness’”
This feels tractable in large part becuase mathematical notation tends to invovle a lot of context, which a language model could probably digest.
Task: convert mathematical expressions into natural language
Context: A researcher is reading a paper about alignment that contains a lot of well-specified but dense mathematical notation. They would like to see a less terse and more fluent description of the same idea that’s easier to read, similar to what a researcher might say to them at a blackboard while writing the math. This might involve additional context for novices.
Input type: a piece of mathematically-dense but well-specified text from a paper
Output type: a fluent, natural language descirption of the same mathematical objects
Info constraints: none
Instance 1:
Input: the section “The circuit distillation prior” from https://www.alignmentforum.org/posts/7ygmXXGjXZaEktF6M/towards-a-better-circuit-prior-improving-on-elk-state-of-the
Output: Consider a predictive model that predicts the output of a video camera given some sensors in the world. As in ELK, our goal will be to find a function that looks at the sensors, the model, and some questions, then reutrns some answers to those questions using the model’s latent knoweldge.
Instance 2:
Input: the section “Our model of proxy misspecification” from https://www.alignmentforum.org/posts/tWpgtjRm9qwzxAZEi/proxy-misspecification-and-the-capabilities-vs-value
Output: Alice has n things that she values: given any of these items, she’ll always value a set at least as much if she adds another one. A robot is given a proxy for this utility, but it dpeends on a strict subset of the items. The robot optimizes its proxy subjec to some resource constrants. It’s a thoerem that the robot will not pick the things that weren’t included in its utility.
Instance 3:
Input: The paragraph “vector valued preferences” from https://www.alignmentforum.org/posts/oheKfWA7SsvpK7SGp/probability-is-real-and-value-is-complex
Output: We think of events, in the sense of the sigma algebras that are used in the formalization fo probability theory. Each event has a probability and an expected utility assigned to it. We are interested in the product of these two, which Valdimir Nesov called “shouldness’”
This feels tractable in large part becuase mathematical notation tends to invovle a lot of context, which a language model could probably digest.
I love this idea mostly because it would hugely improve screen reader options for alignment research.