Better algorithms. Mathematical insights can reduce the computation time of a program by many orders of magnitude without additional hardware. For example, IBM’s Deep Blue played chess at the level of world champion Garry Kasparov in 1997 using about 1.5 trillion instructions per second (TIPS), but a program called Deep Junior did it in 2003 using only 0.15 TIPS. Thus, the power of the chess algorithms increased by a factor of 100 in only six years, or 3.33 orders of magnitude per decade (Richard and Shaw 2004). [add sentence about how this sort of improvement is not uncommon, with citations]
One good example is linear programming and related algorithms. Kaj discussed this earlier here(pdf):
In the past, improvements in algorithms have sometimes been even more important than
improvements in hardware. The President’s Council of Advisors on Science and Technology [2010]
mentions that performance on a benchmark production planning model improved by a factor of 43
million between 1988 and 2003. Out of the improvement, a factor of roughly 1,000 was due to
better hardware and a factor of roughly 43,000 was due to improvements in algorithms. Also
mentioned is an algorithmic improvement of roughly 30,000 for mixed integer programming
between 1991 and 2008.
One good example is linear programming and related algorithms. Kaj discussed this earlier here(pdf):