- Have you tried PyPy? It might increase the number of cases where python is good enough.
- Why C instead of C++? (I assume Rust would slow down prototyping speed because of the borrow checker). Is it because you’re more familiar with it from your side projects?
I haven’t tried PyPy for this particular project, but my experience previously had been that while I usually got a bit of speedup it wasn’t typically much.
Why C instead of C++?
I used to work in C++ and know it pretty well, but I don’t really like it very much and there isn’t anything in C++ I’ve needed here.
I haven’t tried PyPy for this particular project, but my experience previously had been that while I usually got a bit of speedup it wasn’t typically much.
That’s also my experience in most cases, but in others can be much faster. It does especially well on code with lots of looping that can be JITted. As a data point, https://github.com/jeffkaufman/kmer-egd/blob/main/count-quality.py is ~3x faster on my machine (takes 0.8 seconds vs 3 seconds on 100k lines of length 151 containing FF). Which probably is not enough to make a difference, but might still be useful.
You can also use Numba to speed up loops. It’s still slower than C, but it’s much better than plain Python code, and it’s really easy to implement (just import numba and put a @numba.njit() before your function).
Probably dumb questions:
- Have you tried PyPy? It might increase the number of cases where python is good enough.
- Why C instead of C++? (I assume Rust would slow down prototyping speed because of the borrow checker). Is it because you’re more familiar with it from your side projects?
I haven’t tried PyPy for this particular project, but my experience previously had been that while I usually got a bit of speedup it wasn’t typically much.
I used to work in C++ and know it pretty well, but I don’t really like it very much and there isn’t anything in C++ I’ve needed here.
Thanks for the reply!
That’s also my experience in most cases, but in others can be much faster. It does especially well on code with lots of looping that can be JITted.
As a data point, https://github.com/jeffkaufman/kmer-egd/blob/main/count-quality.py is ~3x faster on my machine (takes 0.8 seconds vs 3 seconds on 100k lines of length 151 containing FF).
Which probably is not enough to make a difference, but might still be useful.
Yes, probably not enough to make a difference; that one in particular is fast enough in python. But useful to have the number!
You can also use Numba to speed up loops. It’s still slower than C, but it’s much better than plain Python code, and it’s really easy to implement (just
import numbaand put a@numba.njit()before your function).