See also MultiOn and Maisa. Both are different agent enhancements for LLMs that claim notable new abilities on benchmarks. MultiOn can do web tasks, Maisa scores better on reasoning tasks than COT prompting and uses more efficient calls for lower cost. Neither is in deployment yet, neither company exains exactly how they’re engineered. Ding! Ding!
I also thought developing agents was taking too long until talking to a few people actually working on them. LLMs include new types of unexpected behavior, so engineering around that is a challenge. And there’s the standard time to engineer anything reliable and usable enough to be useful.
So, we’re right on track for language model cognitive architectures with alarmingly fast timelines, coupled with a slow enough takeoff that we’ll get some warning shots.
Edit: I just heard about another one, GoodAI, developing the episodic (long term) memory that I think will be a key element of LMCA agents. They outperform 128k context GPT4T with only 8k of context, on a memory benchmark of their own design, at 16% of the inference cost. Thanks, I hate it.
Edit: I just heard about another one, GoodAI, developing the episodic (long term) memory that I think will be a key element of LMCA agents. They outperform 128k context GPT4T with only 8k of context, on a memory benchmark of their own design, at 16% of the inference cost. Thanks, I hate it.
GoodAI’s Web site says they’re working on controlling drones, too (although it looks like a personal pet project that’s probably not gonna go that far). The fun part is that their marketing sells “swarms of autonomous surveillance drones” as “safety”. I mean, I guess it doesn’t say killer drones...
Very little alignment work of note, despite tons of published work on developing agents. I’m puzzled as to why the alignment community hasn’t turned more of their attention toward language model cognitive architectures/agents, but I’m also reluctant to publish more work advertising how easily they might achieve AGI.
ARC Evals did set up a methodology for Evaluating Language-Model Agents on Realistic Autonomous Tasks. I view this as a useful acknowledgment of the real danger of better LLMs, but I think it’s inherently inadequate, because it’s based on the evals team doing the scaffolding to make the LLM into an agent. They’re not going to be able to devote nearly as much time to that as other groups will down the road. New capabilities are certainly going to be developed by combinations of LLM improvements, and hard work at improving the cognitive architecture scaffolding around them.
I think evals are fantastic (ie obviously a good and correct thing to do; dramatically better than doing nothing) but there is a little bit of awkwardness in terms of deciding how hard to try. You don’t really want to spend a well-funded-startup’s worth of effort to trigger dangerous capabilities (and potentially cause your own destruction), but you know eventually that someone will. I don’t know how to resolve this.
Totally agree with you here. I think probably half of their development energy was spent getting to where GPT-4 Functions were right when Functions came out and they were probably like...oh...welp.
See also MultiOn and Maisa. Both are different agent enhancements for LLMs that claim notable new abilities on benchmarks. MultiOn can do web tasks, Maisa scores better on reasoning tasks than COT prompting and uses more efficient calls for lower cost. Neither is in deployment yet, neither company exains exactly how they’re engineered. Ding! Ding!
I also thought developing agents was taking too long until talking to a few people actually working on them. LLMs include new types of unexpected behavior, so engineering around that is a challenge. And there’s the standard time to engineer anything reliable and usable enough to be useful.
So, we’re right on track for language model cognitive architectures with alarmingly fast timelines, coupled with a slow enough takeoff that we’ll get some warning shots.
Edit: I just heard about another one, GoodAI, developing the episodic (long term) memory that I think will be a key element of LMCA agents. They outperform 128k context GPT4T with only 8k of context, on a memory benchmark of their own design, at 16% of the inference cost. Thanks, I hate it.
GoodAI’s Web site says they’re working on controlling drones, too (although it looks like a personal pet project that’s probably not gonna go that far). The fun part is that their marketing sells “swarms of autonomous surveillance drones” as “safety”. I mean, I guess it doesn’t say killer drones...
Any new safety studies on LMCA’s?
Very little alignment work of note, despite tons of published work on developing agents. I’m puzzled as to why the alignment community hasn’t turned more of their attention toward language model cognitive architectures/agents, but I’m also reluctant to publish more work advertising how easily they might achieve AGI.
ARC Evals did set up a methodology for Evaluating Language-Model Agents on Realistic Autonomous Tasks. I view this as a useful acknowledgment of the real danger of better LLMs, but I think it’s inherently inadequate, because it’s based on the evals team doing the scaffolding to make the LLM into an agent. They’re not going to be able to devote nearly as much time to that as other groups will down the road. New capabilities are certainly going to be developed by combinations of LLM improvements, and hard work at improving the cognitive architecture scaffolding around them.
I think evals are fantastic (ie obviously a good and correct thing to do; dramatically better than doing nothing) but there is a little bit of awkwardness in terms of deciding how hard to try. You don’t really want to spend a well-funded-startup’s worth of effort to trigger dangerous capabilities (and potentially cause your own destruction), but you know eventually that someone will. I don’t know how to resolve this.
Totally agree with you here. I think probably half of their development energy was spent getting to where GPT-4 Functions were right when Functions came out and they were probably like...oh...welp.