Nice story! Mostly I think that the best AGIs will always be in the big labs rather than open source, and that current open-source models aren’t smart enough to get this sort of self-improving ecosystem off the ground. But it’s not completely implausible.
Thank you very much! I agree. We chose this scenario out of many possibilities because so far it hasn’t been described in much detail and because we wanted to point out that open source can also lead to dangerous outcomes, not because it is the most likely scenario. Our next story will be more “mainstream”.
Nice story! Mostly I think that the best AGIs will always be in the big labs rather than open source, and that current open-source models aren’t smart enough to get this sort of self-improving ecosystem off the ground. But it’s not completely implausible.
Thank you very much! I agree. We chose this scenario out of many possibilities because so far it hasn’t been described in much detail and because we wanted to point out that open source can also lead to dangerous outcomes, not because it is the most likely scenario. Our next story will be more “mainstream”.