Part 1/8:
Understanding Large Language Models: How Facts are Stored
Large language models (LLMs) such as GPT-3 have revolutionized the field of artificial intelligence by enabling machines to understand and generate human-like text. A key question that arises in the exploration of these models is: how do they store and retrieve facts? One interesting illustration of this concept involves the statement "Michael Jordan plays the sport of blank." When given such an input, a well-trained model will predict "basketball," suggesting that the knowledge about Michael Jordan and his association with basketball is embedded somewhere within its parameters.