{"id":2098,"date":"2025-02-27T07:02:52","date_gmt":"2025-02-27T07:02:52","guid":{"rendered":"https:\/\/mailitics.com\/index.php\/2025\/02\/27\/llada-the-diffusion-model-that-could-redefine-language-generation\/"},"modified":"2025-02-27T07:02:52","modified_gmt":"2025-02-27T07:02:52","slug":"llada-the-diffusion-model-that-could-redefine-language-generation","status":"publish","type":"post","link":"https:\/\/mailitics.com\/index.php\/2025\/02\/27\/llada-the-diffusion-model-that-could-redefine-language-generation\/","title":{"rendered":"LLaDA: The Diffusion Model That Could Redefine Language Generation"},"content":{"rendered":"

LLaDA: The Diffusion Model That Could Redefine Language Generation
\n \t

\n
<\/BR>
\n
\n \t

\n
<\/BR><\/p>\n

\n

Introduction<\/h2>\n

What if we could make language models think\u00a0more like humans<\/strong>? Instead of writing one word at a time, what if they could sketch out their thoughts first, and gradually refine them?<\/p>\n

This is exactly what Large Language Diffusion Models<\/a> (LLaDA) introduces: a different approach to current text generation used in Large Language Models (LLMs). Unlike traditional autoregressive models (ARMs), which predict text sequentially, left to right,\u00a0LLaDA leverages a diffusion-like process to generate text.<\/strong>\u00a0Instead of generating tokens sequentially, it\u00a0progressively refines masked text until it forms a coherent response<\/strong>.<\/p>\n

In this article, we will dive into how LLaDA works, why it matters, and how it could shape the next generation of LLMs.<\/p>\n

I hope you enjoy the article!<\/p>\n

The current state of LLMs<\/h2>\n

To appreciate the innovation that LLaDA represents, we first need to understand how current large language models (LLMs) operate. Modern LLMs follow a two-step training process that has become an industry standard:<\/p>\n

    \n
  1. \nPre-training<\/strong>: The model learns general language patterns and knowledge by predicting the next token in massive text datasets through self-supervised learning.<\/li>\n
  2. \nSupervised Fine-Tuning (SFT)<\/strong>: The model is refined on carefully curated data to improve its ability to follow instructions and generate useful outputs.<\/li>\n<\/ol>\n

    Note that current LLMs often use RLHF as well to further refine the weights of the model, but this is not used by LLaDA so we will skip this step here.<\/em><\/p>\n

    These models, primarily based on the Transformer architecture, generate text\u00a0one token at a time<\/strong>\u00a0using next-token prediction.<\/p>\n

    \"\"
    Simplified Transformer architecture for text generation (Image by the author)<\/figcaption><\/figure>\n

    Here is a simplified illustration of how data passes through such a model.\u00a0Each token is embedded into a vector and is transformed through successive transformer layers<\/strong>. In current LLMs (LLaMA, ChatGPT, DeepSeek, etc), a classification head is used only on the last token embedding to predict the next token in the sequence.<\/p>\n

    This works thanks to the concept of\u00a0masked self-attention<\/strong>: each token attends to all the tokens that come before it. We will see later how LLaDA can get rid of the mask in its attention layers.<\/p>\n

    \"\"
    Attention process: input embeddings are multiplied byQuery, Key, and Value matrices to generate new embeddings (Image by the author, inspired by [3])<\/figcaption><\/figure>\n

    If you want to learn more about Transformers, check out my article here<\/a><\/em>.<\/p>\n

    While this approach has led to impressive results, it also comes with significant limitations, some of which have motivated the development of LLaDA.<\/p>\n

    Current limitations of LLMs<\/h2>\n

    Current LLMs face several critical challenges:<\/p>\n

    Computational Inefficiency<\/h3>\n

    Imagine having to write a novel where\u00a0you can only think about one word at a time,<\/strong>\u00a0and for each word, you need to reread everything you\u2019ve written so far. This is essentially how current LLMs operate \u2014 they predict one token at a time, requiring a complete processing of the previous sequence for each new token. Even with optimization techniques like KV caching, this process is\u00a0quite computationally expensive and time-consuming<\/strong>.<\/p>\n

    Limited Bidirectional Reasoning<\/h3>\n

    Traditional autoregressive models (ARMs) are like writers who could never look ahead or revise what they\u2019ve written so far.\u00a0They can only predict future tokens based on past ones, which limits their ability to reason about relationships between different parts of the text.<\/strong>\u00a0As humans, we often have a general idea of what we want to say before writing it down, current LLMs lack this capability in some sense.<\/p>\n

    Amount of data<\/h3>\n

    Existing models require\u00a0enormous amounts of training data<\/strong>\u00a0to achieve good performance, making them resource-intensive to develop and potentially limiting their applicability in specialized domains with limited data availability.<\/p>\n

    What is LLaDA<\/h2>\n

    LLaDA introduces a fundamentally different approach to Language Generation<\/a> by replacing traditional autoregression with a\u00a0\u201cdiffusion-based\u201d<\/strong>\u00a0process (we will dive later into why this is called \u201cdiffusion\u201d).<\/p>\n

    Let\u2019s understand how this works, step by step, starting with pre-training.<\/p>\n

    LLaDA pre-training<\/h3>\n

    Remember that we don\u2019t need any \u201clabeled\u201d data during the pre-training phase. The objective is to feed a very large amount of raw text data into the model. For each text sequence, we do the following:<\/p>\n

      \n
    1. We fix a maximum length (similar to ARMs). Typically, this could be 4096 tokens. 1% of the time, the lengths of sequences are randomly sampled between 1 and 4096 and padded so that the model is also exposed to shorter sequences.<\/li>\n
    2. We randomly choose a \u201cmasking rate\u201d. For example, one could pick 40%.<\/li>\n
    3. We mask each token with a probability of 0.4. What does \u201cmasking\u201d mean exactly? Well, we simply replace the token\u00a0with a special token<\/strong>:\u00a0<MASK><\/strong>. As with any other token, this token is associated with a particular index and embedding vector that the model can process and interpret during training.<\/li>\n
    4. We then feed our entire sequence into our transformer-based model. This process transforms all the input embedding vectors into new embeddings. We\u00a0apply the classification head to each of the masked tokens<\/strong>\u00a0to get a prediction for each. Mathematically, our loss function averages cross-entropy losses over all the masked tokens in the sequence, as below:<\/li>\n<\/ol>\n
      \"\"
      Loss function used for LLaDA (Image by the author)<\/figcaption><\/figure>\n

      5. And\u2026 we repeat this procedure for billions or trillions of text sequences.<\/p>\n

      \n

      Note, that unlike ARMs, LLaDA can fully utilize bidirectional dependencies in the text: it doesn\u2019t require masking in attention layers anymore. However, this can come at an increased computational cost.<\/em><\/strong><\/p>\n<\/blockquote>\n

      Hopefully, you can see how the training phase itself (the flow of the data into the model) is very similar to any other LLMs.\u00a0We simply predict randomly masked tokens instead of predicting what comes next.<\/strong><\/p>\n

      LLaDA SFT<\/h3>\n

      For\u00a0auto-regressive models<\/strong>, SFT is very similar to pre-training, except that we have pairs of\u00a0(prompt, response)\u00a0<\/em>and want to generate the response when giving the prompt as input.<\/p>\n

      This is exactly the\u00a0same concept for LlaDa<\/a><\/strong>! Mimicking the pre-training process: we simply pass the prompt and the response, mask random tokens\u00a0from the response only<\/strong>, and feed the full sequence into the model, which\u00a0will predict missing tokens from the response<\/strong>.<\/p>\n

      The innovation in inference<\/h3>\n

      Innovation is where LLaDA gets more interesting, and truly utilizes the \u201cdiffusion\u201d paradigm.<\/p>\n

      Until now, we always randomly masked some text as input and asked the model to predict these tokens. But during inference,\u00a0we only have access to the prompt\u00a0<\/strong>and we need to generate the entire response. You might think (and it\u2019s not wrong), that the model has seen examples where the masking rate was very high (potentially 1) during SFT, and it had to learn, somehow, how to\u00a0generate a full response from a prompt<\/strong>.<\/p>\n

      However, generating the full response at once during inference will likely produce very poor results because the model lacks information. Instead, we need a method to\u00a0progressively refine predictions<\/strong>, and that\u2019s where the key idea of\u00a0\u2018remasking\u2019\u00a0<\/strong>comes in.<\/p>\n

      Here is how it works, at each step of the text generation process:<\/p>\n