Teachers

How LLMs really work

When you ask an AI chatbot to fix your code or explain a concept, it can feel like you're talking to something that understands you. But under the hood, an LLM is doing something much simpler and much stranger. It is not magic - it's math.

At its core, an LLM is a very large mathematical function that takes your prompt as input and calculates, token by token, what the most probable response looks like. No reasoning, no understanding, no knowledge base being queried. Just pattern matching at massive scale that produces something that looks like intelligence. Once you understand that, you'll have a much clearer picture of why LLMs are so capable in some situations, and so confidently wrong in others.

An honest introduction to an LLM would be:

"Hi I am ChatGPT. I am a 1 terabyte zip file. My knowledge comes from the internet, which I read in its entirety about 6 months ago and remember only vaguely. My winning personality was programmed, by example, by human labelers at OpenAI :)"

Source: How I use LLMs by Andrej Karpathy [12:15]

Watch: A quick overview of LLMs and chatbots under the hood

<aside> 💭

Watch the following video up until minute 13:13.

</aside>

https://youtu.be/EWvNQjAaOHw?si=76BFNraP2qBaSfMK&t=164

Key takeaways from the video

LLMs don't input and output words or text, they work with very large arrays of numbers (tokens) instead.
The process of converting a string to this large array of numbers (tokens) is called tokenization. We will cover this topic in a later chapter.
Both the AI and the human add more and more tokens to the same large array of numbers, which is called the context window.

How an LLM is trained (In a nutshell)

Training an LLM is a complicated, lengthy, and expensive process. It requires vast amounts of data and powerful computers to process this information into a usable model. In this section, we'll briefly explain the key stages and terms involved in creating an LLM.

Pre-training is the first and most expensive stage. The model is fed an enormous amount of text: websites, books, code, articles and learns to predict the next token. This is where the model acquires its general knowledge of language, facts, and patterns. It takes months and costs millions of dollars in compute. The result is a model that is knowledgeable but raw - it predicts text, it doesn't yet behave like a helpful assistant.
Post-training is where the raw model gets shaped into something useful. This mainly involves two steps:
- Fine-tuning the model is trained on selected examples of good conversations, teaching it to respond in a helpful, structured way
- RLHF (Reinforcement Learning from Human Feedback) - human raters score model responses, and the model is adjusted to produce outputs that humans prefer.

<aside> 💭

The result is the assistant you interact with: knowledgeable from pre-training, well-behaved from post-training. Pre-training gives the model its knowledge, post-training gives it its personality.

</aside>

Additional Resources

Video

The HackYourFuture curriculum is licensed under CC BY-NC-SA 4.0

CC BY-NC-SA 4.0 Icons

*https://hackyourfuture.net/*

Found a mistake or have a suggestion? Let us know in the feedback form.

Week 11 - OOP concepts & LLMs