Bjrn answers What were the biggest discoveries /​ innovations in AI and ML?

[Bjrn]

From A ‘Brief’ History of Neural Nets and Deep Learning, Part 4:

So, why indeed, did purely supervised learning with backpropagation not work well in the past? Geoffrey Hinton summarized the findings up to today in these four points:

1. Our labeled datasets were thousands of times too small.

2. Our computers were millions of times too slow.

3. We initialized the weights in a stupid way.

4. We used the wrong type of non-linearity.

I think this blog series might help provide a partial answer to your question.