Adria (other coauthor on the paper) told me that Redwood ported ACDC to Rust early on, which did provide a useful speedup (idk how much but my guess is 10-100x?) but made it harder to maintain. I’m currently working in Python. I wouldn’t believe those marketing numbers for anything but the Mandelbrot task they test it on, which has particularly high interpreter overhead.
The bigger problem with ACDC is it doesn’t use gradients. Attribution patching fixes this and gets >100x speedups, and there should be even better methods. I don’t expect circuit discovery to be usefully ported to a fast language again, until it is used in production.
I do not understand the appeal of MOJO. If you’re going to overcomplicate Python to the point of C++, just use C++. Or, you know, a saner systems language like Rust. CPython has C interop already.
For what it’s worth, I’ve had to drop from python to C# on occasion for some bottlenecks. In one case, my C# implementation was 418,000 times faster than the python version. That’s a comparison between a poor python implementation and a vectorized C# implementation, but… yeah.
I’m curious to know how much the code could be faster through using a faster programming language. For example, MOJO. @Arthur Conmy
Adria (other coauthor on the paper) told me that Redwood ported ACDC to Rust early on, which did provide a useful speedup (idk how much but my guess is 10-100x?) but made it harder to maintain. I’m currently working in Python. I wouldn’t believe those marketing numbers for anything but the Mandelbrot task they test it on, which has particularly high interpreter overhead.
The bigger problem with ACDC is it doesn’t use gradients. Attribution patching fixes this and gets >100x speedups, and there should be even better methods. I don’t expect circuit discovery to be usefully ported to a fast language again, until it is used in production.
I do not understand the appeal of MOJO. If you’re going to overcomplicate Python to the point of C++, just use C++. Or, you know, a saner systems language like Rust. CPython has C interop already.
For what it’s worth, I’ve had to drop from python to C# on occasion for some bottlenecks. In one case, my C# implementation was 418,000 times faster than the python version. That’s a comparison between a poor python implementation and a vectorized C# implementation, but… yeah.