Alec Radford and the OpenAI team deploy the first Generative Pre-trained Transformer, applying autoregressive self-attention to massive text corpora. This design demonstrates that raw language modeling scales into general task capabilities.
Part of the 29 Structural Foundations: The Internet Era, Big Data, and Deep Learning Explosion Edition archive. HistoricallyVerified
Top 5 Structural Foundations: Origins
- The Deployment of Machine Learning for Google Search RankBrain — Google deployed RankBrain, a deep learning text-embedding system designed to interpret ambiguous sea...
- The Introduction of the Deep Boltzmann Machine Optimization — Ruslan Salakhutdinov and Geoffrey Hinton formalized advanced learning procedures for deep Boltzmann ...
- The Arrival of human-Level Performance on ImageNet — Researchers at Microsoft Research and Google independently announced that their deep convolutional n...
- The First Viral “Meme” (Dancing Baby) — A 3D-rendered animation of a Dancing Baby (Oogachacka) spreads rapidly via email and web forums, be...
- Ars Magna and Algorithmic Faith — Around 1300, Majorcan philosopher Ramon Llull designed a system of rotating, concentric mechanical w...
Discussion: