ProblemFlorence-2 originally supports three attention backends: eager (manual), sdpa (PyTorch Scaled Dot-Product Attention), and flash_attention_2 (Flash Attention 2). While these are functional, newer quantized attention kernels - SageAttention2 and SageAttention3 - offer significant speedups (2-5x over FlashAttention) with negligible accuracy loss. SolutionAdded sage_attention_2 and sage_attention_3 as selectable attention backends in the ComfyUI UI.
Architecture and base class rationale
Modified Files1. ...Why a base class? Both SA2 and SA3 share identical Q/K/V projection and output reshaping logic. The only difference is which function computes the attention scores. The base class unifies these shared parts, enabling a clean substitution of the scoring function while preserving the rest of the pipeline.
Fallback to SDPA when attention_mask is present - SageAttention does not support arbitrary attention masks. When the encoder receives padded inputs (batched inference), the mask ensures padding tokens are ignored. No pre-scaling of query states - Unlike the eager Florence2Attention which pre-multiplies queries by head_dim ** -0.5, both SDPA and SageAttention handle scaling internally. The is_causal flag - Passed through to SageAttention. The encoder uses is_causal=False (bidirectional), the decoder uses is_causal=True (autoregressive).
The return sageattnUses the sageattention package (INT8 quantized Q/K with FP16/FP8 PV accumulation). The return sageattn3_blackwellUses the sageattn3 package (INT4/FP4 quantized attention optimized for NVIDIA Blackwell architecture).
This dictionary is used by Florence2EncoderLayer and Florence2DecoderLayer to instantiate the correct attention class based on config._attn_implementation.
Why and how this integration works
config._attn_implementation = attentionWhy this is necessary: Transformers 5.x validates the _attn_implementation value in PreTrainedModel.__init__() against a whitelist (eager, sdpa, flash_attention_2, etc.). Custom values like sage_attention_2 are rejected with a ValueError. print(f"Florence2 attention replaced with {sage_mode}")
How this works: After the model is fully initialized and weights are loaded, we iterate through all modules and replace the Python class of each SDPA attention layer with the corresponding SageAttention class. print(f"{attention} requires transformers>=5.0, falling back to sdpa")
For users on transformers 4.x, SageAttention is not available (the v5 load_model path is required for the class-swap technique).
Installation and compatibility considerations
Why Does SageAttention Fail to Install in ComfyUI?Quick answer: SageAttention installs fail in ComfyUI almost always for one of four reasons: a version mismatch between Python, CUDA, PyTorch, and Triton; a missing Triton include/libs copy; the wrong wheel file for your setup; or an outdated PyTorch build. Fix them in this exact order - Visual C++ → Triton → PyTorch → the SageAttention wheel - and the install succeeds.
SageAttention is an attention kernel - a piece of code that speeds up one of the slowest calculations ComfyUI runs during image and video generation. It does not change what your images look like; it only makes that one calculation faster, which is why the install itself is where almost all the problems happen.
Before jumping into the steps, it helps to know exactly what breaks the install. In order of how often each one shows up: A version mismatch - Python, CUDA, PyTorch, and Triton all need matching versions. The SageAttention wheel file (a pre-built Python package ending in .whl) is compiled for one specific combination of these - not "any recent version." Triton's include/libs files not copied into python_embeded - ComfyUI's embedded Python is missing these development files by default, and Triton cannot compile without them. The wrong wheel downloaded - the ABI tag (the abi3 part of the filename) and CUDA version in the filename must match what you have installed, or pip refuses the install outright. An outdated PyTorch build - SageAttention needs a PyTorch build compiled against the correct CUDA version, not just any PyTorch you already had installed.
This guide walks through all four in the order that actually works, with the exact commands and version checks at each step.
What you need before installing
- Supported GPUs: NVIDIA RTX 20xx, 30xx, 40xx, or 50xx series. RTX 50xx requires Triton ≥ 3.3, PyTorch ≥ 2.7, and CUDA ≥ 12.8.
- Operating System: Windows 10 or Windows 11 with the ComfyUI Windows portable version already installed.
- Required Packages: Visual C++ Redistributable, triton-windows, torch + torchvision + torchaudio, and the SageAttention .whl file.
Version matching and preparation
All four components must have compatible versions - Python, CUDA, Triton, and PyTorch must all match the SageAttention wheel you download. It also helps to update ComfyUI to the latest version before starting. Older ComfyUI portable builds sometimes ship an outdated bundled PyTorch that conflicts with the CUDA 12.8 build SageAttention needs.
Step-by-step installation outline
- Step 1 - Install Visual C++ Redistributable: Run the installer, click Install or Repair when prompted, and restart Windows if required.
- Step 2 - Install Triton (triton-windows): Open a terminal in the ComfyUI Windows portable folder. Remove any old Triton, then install the Windows-compatible Triton build. Copy the include and libs files matching your embedded Python version into the python_embeded folder to allow Triton to load.
- Step 3 - Install or Update PyTorch: Use a PyTorch build that matches your CUDA version and the wheel you will install.
- Step 4 - Download and Install the Correct SageAttention Wheel: Check CUDA and Torch versions, then download the wheel that matches (examples provided for cu128/cu130). Place the wheel in the ComfyUI portable folder and install with pip using the exact filename.
- Step 5 - Verify SageAttention Is Working: Start ComfyUI, observe the console for SageAttention and Triton loading messages, and run a simple workflow to confirm speed improvements without changes to output.
Working Setup Example (reference): OS Windows 11, GPU RTX 4090, Python 3.13.11 (embedded), Triton Windows 3.5.1.post24, PyTorch 2.12.0.dev20260219+cu128, SageAttention wheel sageattention-2.2.0+cu128torch2.9.0andhigher.post4-cp39-abi3-win_amd64.whl.
Does SageAttention Slow Down or Change Image Quality? No. SageAttention only speeds up the attention calculation - it does not change the output. When installed correctly, you should see a real drop in generation time with no visible difference in the result. In many cases, generation time drops by 30-50%, with larger gains for heavier workloads such as FLUX models.
Practical considerations for use with ComfyUI
ProblemFlorence-2 originally supports three attention backends: eager (manual), sdpa, and flash_attention_2. SageAttention2 and SageAttention3 offer substantial speedups (2-5x over FlashAttention) with negligible accuracy loss. SolutionAdded sage_attention_2 and sage_attention_3 as selectable backends in the ComfyUI UI.
Why a base class? Both SA2 and SA3 share identical Q/K/V projection and output reshaping logic; only the function that computes attention scores differs. This enables a unified switch in the UI between SageAttention backends while preserving the rest of Florence2’s architecture. Fallback to SDPA when attention_mask is present, since SageAttention does not support arbitrary masks. The encoder uses is_causal=False and the decoder uses is_causal=True. The implementation uses the sageattention package for SA2 (INT8 Q/K with FP16/FP8 PV accumulation) and the sageattn3 package for SA3 (INT4/FP4 optimized for Blackwell). This setup is used by Florence2EncoderLayer and Florence2DecoderLayer to instantiate the correct attention class based on config._attn_implementation.
In Transformers 5.x, the _attn_implementation value must be whitelisted; custom values like sage_attention_2 are rejected in older versions, so the system replaces the SDPA attention at runtime with the SageAttention class after loading weights. For users on Transformers 4.x, SageAttention is not available and requires the v5 load_model path for the class-swap technique.
Visual and multimedia enhancement options
To maximize understanding, consider these optional visual aids:

The following video can illustrate the concept of attention kernels and the speed differences between backends:
Attention in transformers, step-by-step | Deep Learning Chapter 6
If helpful, a compact table can summarize the core differences between backends, though the article remains focused on the narrative of SageAttention integration and operational impact.
Includes and notes on usage
Use the SageAttention backends via the ComfyUI UI selection. The base-class approach keeps the rest of Florence2's behavior intact while enabling substantial speedups in the attention computation stage.