Personally, I have some long lists of ideas for things I haven’t got time for including: research projects in AI, research projects in other subjects which could be advanced entirely by work on a computer (e.g. collecting and summarizing relevant facts from papers, running physical simulations of potential designs, etc), games, books, productivity tools, etc.
I’ve tried some of the current AI agent stuff, and nothing I’ve tried is quite good enough with the current set of models to automated enough of actualizing my ideas to make it worth my time. I’m prioritizing saving the lives of everyone on Earth, including everyone I love, by attempting to reduce the risk of AI catastrophe. Maybe next year, the critical point will be reached where spending a lot on inference to make many tries at each necessary step will become effective. If I could just dump in a couple thousand dollars a month into AI agent inference working on my ideas, and get a handful of mostly complete projects out, then I’d be making tons of money even if my success rate for the ideas taking off were 1 in 1000.
If you aren’t the sort of person who does have lists of potentially valuable projects sitting around waiting for intelligent workers to breathe life into them… I dunno. Maybe the next generation of models will be good enough to also help you with the ideation phase?
Maybe next year, the critical point will be reached where spending a lot on inference to make many tries at each necessary step will become effective.
That raises an excellent point that hasn’t been otherwise brought up—it’s clear that there are at least some cases already where you can get much better performance by doing best-of-n with large n. I’m thinking especially of Ryan Greenblatt’s approach to ARC-AGI, where that was pretty successful (n = 8000). And as Ryan points out, that’s the approach that AlphaCode uses as well (n = some enormous number). That seems like plausibly the best use of a lot of money with current LLMs.
If the idea is “you should use AI to work on your personal projects”, the problem is I’m already doing that (notice the $50/month). I’m looking for ways to spend 20x more on AI without spending 20x more of my time (which is physically impossible).
Personally, I have some long lists of ideas for things I haven’t got time for including: research projects in AI, research projects in other subjects which could be advanced entirely by work on a computer (e.g. collecting and summarizing relevant facts from papers, running physical simulations of potential designs, etc), games, books, productivity tools, etc.
I’ve tried some of the current AI agent stuff, and nothing I’ve tried is quite good enough with the current set of models to automated enough of actualizing my ideas to make it worth my time. I’m prioritizing saving the lives of everyone on Earth, including everyone I love, by attempting to reduce the risk of AI catastrophe. Maybe next year, the critical point will be reached where spending a lot on inference to make many tries at each necessary step will become effective. If I could just dump in a couple thousand dollars a month into AI agent inference working on my ideas, and get a handful of mostly complete projects out, then I’d be making tons of money even if my success rate for the ideas taking off were 1 in 1000.
If you aren’t the sort of person who does have lists of potentially valuable projects sitting around waiting for intelligent workers to breathe life into them… I dunno. Maybe the next generation of models will be good enough to also help you with the ideation phase?
That raises an excellent point that hasn’t been otherwise brought up—it’s clear that there are at least some cases already where you can get much better performance by doing best-of-n with large n. I’m thinking especially of Ryan Greenblatt’s approach to ARC-AGI, where that was pretty successful (n = 8000). And as Ryan points out, that’s the approach that AlphaCode uses as well (n = some enormous number). That seems like plausibly the best use of a lot of money with current LLMs.
If the idea is “you should use AI to work on your personal projects”, the problem is I’m already doing that (notice the $50/month). I’m looking for ways to spend 20x more on AI without spending 20x more of my time (which is physically impossible).