Flow matching is a technique for training generative AI models to create images. It offers a formulation related to diffusion processes, with characteristics that can lead to simpler path construction and trajectories that are easier to integrate with relatively few sampling steps. This method is employed in systems such as the one described for Stable Diffusion 3 (Scaling Rectified Flow Transformers for High-Resolution Image Synthesis).

This guide explains the core concepts behind flow matching image generation, how it works from training to sampling, and how it compares to diffusion models.

Key takeaways

  • Flow matching trains a neural network to learn a "velocity field," which is a map of directions that guides random noise toward the structure of a real image.
  • To generate an image, an Ordinary Differential Equation (ODE) solver follows this learned velocity field, transforming a starting noise sample into a final image step-by-step.
  • Rectified Flow, a common variant, uses straight-line paths between noise and images during training. This encourages the model to learn paths that are easier for an ODE solver to follow, enabling generation with fewer steps.
  • Flow matching and diffusion are related continuous-time generative models. Some systems combine a flow-matching objective with other architectures like transformers.
  • While flow matching can facilitate faster sampling, overall performance depends on the model architecture, the ODE solver used, and the complexity of the training data.

What Is Flow Matching in Image Generation?

Flow matching trains a neural network to learn a velocity field that defines a continuous path from a simple noise distribution to a complex image distribution. The model learns how to "flow" random pixels into a structured image. For generation, an Ordinary Differential Equation (ODE) solver follows this path to transform noise into a new image.

Imagine a canvas covered in randomly scattered sand. You want to arrange the sand into a picture of a cat. Flow matching is like learning a set of instructions for a fan that can blow each grain of sand to its correct final position.

The model does not learn the final cat picture directly. Instead, it learns the velocity—the precise direction and speed—that each grain of sand should move at any given moment in time. This set of instructions across the entire canvas is the velocity field.

This process is described mathematically by an Ordinary Differential Equation (ODE). An ODE is a formula that describes how something changes over continuous time. In this case, it describes the continuous transformation of a noisy image into a clear one by following the learned velocities. The foundational flow-matching paper presented this as a way to train models without needing to simulate a full diffusion process during training (Flow Matching for Generative Modeling).

How Does Flow Matching Work, Step by Step?

Flow matching involves a distinct training process to learn the velocity field and a sampling process that uses an ODE solver to generate images. Here is a step-by-step breakdown of how a common variant, Rectified Flow, works.

Training the Velocity Model

The goal of training is to teach a neural network, v_θ, to predict the correct velocity vector for any point along the path between a real image and noise.

  1. Define a Path. Start with a real image from your dataset (x₀) and a sample of random noise (x₁) of the same dimensions. A path is defined between them. For Rectified Flow, this path is a simple straight line determined by a time variable t that goes from 0 to 1. The point on the path at time t is a linear interpolation: xₜ = (1-t)x₀ + tx₁ (Source). At t=0, the point is the data x₀, and at t=1, it is the noise x₁.

  2. Calculate the Target Velocity. For this straight-line path from data to noise, the velocity is constant. It is the vector pointing from the data to the noise: v = x₁ - x₀. This vector is the ground-truth target that the model must learn to predict.

  3. Train the Model. The neural network v_θ(xₜ, t, c) is given the interpolated image xₜ, the current time t, and any optional conditioning information c (like a text prompt). It is trained to output the target velocity v. The loss function measures the difference between the model's prediction and the target, pushing the network to get its prediction right for all points in time and space.

Generating an Image (Sampling)

Once the model is trained, it can generate new images by traversing the path in reverse, starting from pure noise and moving toward data.

  1. Start with Noise. Sample a random tensor z from the same noise distribution used in training. This is your starting point at t=1.

  2. Solve the ODE. Use a numerical ODE solver, such as the Euler method, to trace the path defined by the learned velocity field v_θ. The solver moves backward in time from t=1 to t=0.

    • At each step, the solver asks the model: "Given my current position xₜ and time t, what is the velocity v_θ(xₜ, t, c)?"
    • It then updates the position by taking a small step in the opposite direction of the learned velocity: x_{t-Δt} = xₜ - v_θ(xₜ, t, c) * Δt, where Δt is the step size.
  3. Repeat until t=0. This process is repeated for a set number of steps until the time variable reaches 0. The final x₀ is the generated image. The mechanisms that control these steps are sometimes called diffusion samplers or schedulers.

A Worked Example: Calculating a Velocity Target

The math behind flow matching can be abstract. This is a simplified example for a single grayscale pixel, with values normalized between -1 and 1.

  • Our real data sample x₀ (a light gray pixel) has a value of 0.8.
  • Our random noise sample x₁ has a value of -1.2.

1. Calculate the Target Velocity

The path moves from 0.8 to -1.2. For the straight path used in Rectified Flow, the target velocity is constant: v = x₁ - x₀ = -1.2 - 0.8 = -2.0

The model must learn to predict a velocity of -2.0 for any point along this specific pixel's trajectory from data to noise.

2. Find an Intermediate Point

Let's pick a point in time, say t = 0.25, which is 25% of the way along the path from data to noise. We find the interpolated pixel value x_0.25 using the formula xₜ = (1-t)x₀ + tx₁:

x_0.25 = (1 - 0.25) * 0.8 + 0.25 * (-1.2) x_0.25 = 0.75 * 0.8 + 0.25 * (-1.2) x_0.25 = 0.6 - 0.3 = 0.3

3. Train the Network

During one training step, the model would be given the input value x_0.25 = 0.3 and the time t = 0.25. The model's task is to output a predicted velocity. The loss function would then compare the model's prediction to the ground-truth target of -2.0, and the model's weights would be updated to reduce this error.

This process is repeated millions of times with different pixels, images, noise samples, and time steps until the network becomes a general-purpose velocity predictor.

How Does Flow Matching Differ From Diffusion Models?

Flow matching is an alternative to other diffusion methods, and the two are closely related (Flow Matching for Generative Modeling). Both are continuous-time models that transform noise into data. The primary differences lie in the training objective and the paths they follow.

Here is a comparison of flow matching and a Denoising Diffusion Probabilistic Model (DDPM), a common type of diffusion model.

Feature Flow Matching Typical Diffusion Model (DDPM)
Training Objective Predicts a velocity vector v that points along a path (e.g., from data to noise). Predicts the noise ε that was added to an image, or predicts the original clean image x₀.
Path Formulation Can use flexible paths. Rectified Flow uses straight lines (xₜ = (1-t)x₀ + tx₁). Follows a specific path defined by a fixed noise-addition schedule (variance schedule).
Sampling Method Uses deterministic ODE solvers by default to integrate the learned velocity field. Can use ODE solvers or Stochastic Differential Equation (SDE) solvers.
Training Process Learns the velocity field in a "simulation-free" manner by regressing on a target vector. Can be framed as learning to reverse a simulated forward diffusion process.

While the training objectives differ, the underlying goal is the same: learn a function that can guide a sample from a simple distribution to a complex one. You can learn more about the specifics of the diffusion process in our guide on how diffusion models work. In practice, many modern systems operate in a compressed data format, similar to a latent diffusion model.

What Is Rectified Flow and Why Are Straight Trajectories Faster?

Rectified Flow is a type of flow matching that uses straight-line paths between noise and data during training (Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow). The use of these paths relates to its potential for faster image generation. Straight trajectories can be easier for numerical ODE solvers to follow, allowing for larger integration steps.

Imagine driving from one city to another.

  • A curved, winding road (analogous to some complex generative paths) requires many small, careful steering adjustments. Taking too large of a "step" (driving too fast in a straight line) can lead to leaving the road.
  • A straight highway (the ideal for Rectified Flow) allows covering the distance with fewer, larger steps. One can travel a long distance before needing another adjustment.

In generative modeling, each "adjustment" is a forward pass through the neural network, which is computationally expensive. By training the model on straight-line examples, Rectified Flow encourages the learned global velocity field to be as straight as possible. A straighter field allows an ODE solver to approximate the full trajectory with fewer steps, reducing the total number of network evaluations (NFE) and speeding up generation (What is Flow Matching).

However, assuming this guarantees one-step generation is a common misconception. The path between one noise-data pair is straight by construction, but the overall velocity field the model learns for the entire data distribution is not perfectly straight. Achieving high-quality generation in just a few steps is an optimization goal, not an automatic outcome (Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow).

When to Use Flow Matching (and When Not To)

Flow matching, particularly Rectified Flow, is a technique for building text-to-image models. It's important to understand its practical trade-offs.

When to Consider Flow Matching

  • When sampling speed is a priority. The potential for few-step generation is a primary motivation. Models trained with a rectified flow objective are often optimized for inference with a low number of function evaluations.
  • To use a simulation-free training objective. The model is trained by directly regressing on a target vector field, which can be a more direct objective than learning to reverse a multi-step noising process.
  • When building on modern architectures. Flow matching pairs well with architectures like transformers. Research on large-scale models has shown results by combining a rectified flow objective with a transformer backbone (Scaling Rectified Flow Transformers for High-Resolution Image Synthesis).

Common Pitfalls and When to Be Cautious

  • Performance is not without cost. Training a large flow-matching model is computationally expensive. It requires large datasets, significant GPU resources, and careful tuning, similar to other large-scale generative models.
  • Generation is not always one-step. While few-step sampling is a goal, achieving high image quality often requires more than a single step. Using too few steps can cause discretization errors or incomplete images if the learned trajectory has curvature.
  • Quality depends on more than the objective. The final image quality is a product of the model architecture, training data, conditioning mechanism, and the ODE solver used. Flow matching is one piece of the system. Poor data or a weak model will produce poor results regardless of the training objective (What is Flow Matching).

What to learn next

Sources and further reading