Inception of LLMs - NLP and Neural Networks
Data Science Central, Tuesday, July 16th, 2024
The creation of Large Language Models didn't happen overnight. Remarkably, the first concept of language models started with rule-based systems dubbed Natural Language Processing. These systems follow predefined rules that make decisions and infer conclusions based on text input.
These systems rely on if-else statements processing keyword information and generating predetermined outputs. Think of a decision tree where output is a predetermined response if the input contains X, Y, Z, or none. For example: If the input includes keywords 'mother,' output 'How is your mother?' Else, output, 'Can you elaborate on that?'
The biggest early advancement was neural networks, which were considered when first introduced in 1943 inspired by neurons in human brain function, by mathematician Warren McCulloch. Neural networks even pre-date the term 'artificial intelligence' by roughly 12 years. The network of neurons in each layer is organized in a specific manner, where each node holds a weight that determines its importance in the network. Ultimately, neural networks opened closed doors creating the foundation on which AI will forever be built.