AI for improving human reasoning seems promising; I’m uncertain whether it makes sense to invest in new custom applications, as maybe improvements in models are going to do a lot of the work.
I’m more bullish on investing in exploration of promising workflows and design patterns. As an example, a series of youtube videos and writeups on using O3 as a forecasting aid for grantmaking, with demonstrations. Or a set of examples of using LLMs to aid in productive meetings, with a breakdown of the tech used and social norms that the participants agreed to. - I think these are much cheaper to do in terms for time and money. - A lot of epistemics seems to be HCI bottlenecked. - Good design patterns are easily copyable, which also means they’re probably underinvested in relative to their returns. - Social diffusion of good epistemic practices will not necessarily hapepn as fast as AI improvements. - Improving the AIs themselves to be more truth seeking and provide good advice—with good benchmarks—is another avenue.
I imagine a fellowship for prompt engineers and designers, prize competitions, or perhaps retroactive funding for people who have already developed good patterns.
I like the idea of getting a lot of small examples of clever uses of LLM in the wild, especially by particularly clever/experimental people.
I recently made this post to try to gather some of the techniques common around this community.
One issue that I have though is that I’m really unsure what it looks like to promote neat ideas like these, outside of writing long papers or making semi-viral or at least [loved by a narrow community] projects.
The most obvious way is via X/Twitter. But this often requires building an X audience, which few people are good at. Occasionally particularly neat images/clips by new authors go viral, but it’s tough.
I’d also flag: - It’s getting cheaper to make web applications. - I think EA has seen more success in making blog posts and web apps than we did things like [presenting neat ideas in videos/tweets]. - Often, [simple custom applications] are pretty crucial for actually testing out an idea. You can generate wireframes, but this only tells you a very small amount.
I guess what I’m getting at is that I think [web applications] are likely a major part of the solution—but that we should favor experimenting with many small ones, rather than going all-in on 2-4 ideas or so.
Good points! I agree that actual prototyping is necessary to see if an idea works, and as a demo it can be far more convincing. Especially w/ the decreased cost of building web apps, leveraging them for fast demos of techniques seems valuable.
AI for improving human reasoning seems promising; I’m uncertain whether it makes sense to invest in new custom applications, as maybe improvements in models are going to do a lot of the work.
I’m more bullish on investing in exploration of promising workflows and design patterns. As an example, a series of youtube videos and writeups on using O3 as a forecasting aid for grantmaking, with demonstrations. Or a set of examples of using LLMs to aid in productive meetings, with a breakdown of the tech used and social norms that the participants agreed to.
- I think these are much cheaper to do in terms for time and money.
- A lot of epistemics seems to be HCI bottlenecked.
- Good design patterns are easily copyable, which also means they’re probably underinvested in relative to their returns.
- Social diffusion of good epistemic practices will not necessarily hapepn as fast as AI improvements.
- Improving the AIs themselves to be more truth seeking and provide good advice—with good benchmarks—is another avenue.
I imagine a fellowship for prompt engineers and designers, prize competitions, or perhaps retroactive funding for people who have already developed good patterns.
Happy to see thinking on this.
I like the idea of getting a lot of small examples of clever uses of LLM in the wild, especially by particularly clever/experimental people.
I recently made this post to try to gather some of the techniques common around this community.
One issue that I have though is that I’m really unsure what it looks like to promote neat ideas like these, outside of writing long papers or making semi-viral or at least [loved by a narrow community] projects.
The most obvious way is via X/Twitter. But this often requires building an X audience, which few people are good at. Occasionally particularly neat images/clips by new authors go viral, but it’s tough.
I’d also flag:
- It’s getting cheaper to make web applications.
- I think EA has seen more success in making blog posts and web apps than we did things like [presenting neat ideas in videos/tweets].
- Often, [simple custom applications] are pretty crucial for actually testing out an idea. You can generate wireframes, but this only tells you a very small amount.
I guess what I’m getting at is that I think [web applications] are likely a major part of the solution—but that we should favor experimenting with many small ones, rather than going all-in on 2-4 ideas or so.
Good points! I agree that actual prototyping is necessary to see if an idea works, and as a demo it can be far more convincing. Especially w/ the decreased cost of building web apps, leveraging them for fast demos of techniques seems valuable.