Nitish Srivastava and Geoffrey Hinton introduce the dropout technique, randomly deactivating network nodes during the training phase. This simple architectural modification prevents complex co-adaptation and reduces overfitting in highly parameterized models.
Part of the 29 Structural Foundations: The Internet Era, Big Data, and Deep Learning Explosion Edition archive. HistoricallyVerified
Top 5 Structural Foundations: Origins
- The Year of the Social Architecture — This year proved that web users were no longer passive consumers. By deploying interactive interfac...
- The Nobel Prize Vindication for Deep Learning Pioneers — The Royal Swedish Academy of Sciences awarded the 2024 Nobel Prize in Physics to John Hopfield and G...
- The Theoretical Discovery of the Deep Learning Information Bottleneck — Tishby and Zalishnyak began presenting mathematical theories suggesting that deep neural networks le...
- The Deployment of the Bitcoin Whitepaper and Cryptographic Nodes — Satoshi Nakamoto published the Bitcoin whitepaper. While a blockchain technology, it introduced dece...
- The Introduction of Inductive Logic Programming (ILP) (1991) — Stephen Muggleton formalized ILP, combining inductive machine learning with traditional logic progra...
Discussion: