ChatGPT and Bard have caught the public's attention by storm, but my own feeling is one of déjà vu. Large language models (LLMs) like these are a recent development, but the 1940s marked the first forays into the technology, when writer Jorge Luis Borges interrogated the nature of generated text and mathematician-engineer Claude Shannon experimented with the simplest language models. Seeing LLMs in light of their work clarifies some of the fundamental questions now before us.
Share this post
Large Language Models of Babel
Share this post
ChatGPT and Bard have caught the public's attention by storm, but my own feeling is one of déjà vu. Large language models (LLMs) like these are a recent development, but the 1940s marked the first forays into the technology, when writer Jorge Luis Borges interrogated the nature of generated text and mathematician-engineer Claude Shannon experimented with the simplest language models. Seeing LLMs in light of their work clarifies some of the fundamental questions now before us.