Tiled inference for upscaling images larger than VRAM: a working pattern

Most upscaling models are trained on 256-512px crops but people want to run them on 4K photos. The model OOMs, so you tile. The naive tiling loop has two failure modes that are easy to miss.

Failure mode 1: seam artifacts from independent normalization

If each tile is normalized separately (per-tile mean/std, or per-tile min/max), the tiles come back at slightly different brightness and you get a visible grid. Normalize over the whole image first, then tile. Obvious in hindsight, but it bites because the per-tile artifacts are subtle in a single tile and obvious in the composite.

Failure mode 2: edge tiles padded with zeros

Zero padding at the borders makes the model hallucinate dark edges. Reflection padding is better but you must crop it back correctly. Getting the crop math wrong by even one pixel produces a one-pixel dark line at every seam, which at 4x looks like a 4px band.

The pattern that works for me:

import torch
import torch.nn.functional as F

@torch.inference_mode()
def tiled_upscale(model, img, scale=4, tile=256, overlap=32):
    # img: (1, C, H, W) float tensor, already normalized over the FULL image
    _, C, H, W = img.shape
    out = torch.zeros((1, C, H * scale, W * scale), dtype=img.dtype, device=img.device)
    weight = torch.zeros_like(out)

    step = tile - 2 * overlap
    for y in range(0, H, step):
        for x in range(0, W, step):
            y0, x0 = max(0, y - overlap), max(0, x - overlap)
            y1, x1 = min(H, y + tile + overlap), min(W, x + tile + overlap)
            patch = img[:, :, y0:y1, x0:x1]

            ph = (32 - patch.shape[2] % 32) % 32
            pw = (32 - patch.shape[3] % 32) % 32
            if ph or pw:
                patch = F.pad(patch, (0, pw, 0, ph), mode='reflect')

            up = model(patch)
            up = up[:, :, : (y1 - y0) * scale, : (x1 - x0) * scale]

            oy0, ox0 = y0 * scale, x0 * scale
            oy1, ox1 = oy0 + up.shape[2], ox0 + up.shape[3]
            out[:, :, oy0:oy1, ox0:ox1] += up
            weight[:, :, oy0:oy1, ox0:ox1] += 1.0

    return out / weight.clamp_min(1.0)

Averaging overlapping regions by weight (instead of last-write-wins) removes the remaining faint seams. weight.clamp_min(1) guards against holes if your step math ever leaves a gap.

Memory notes

  • tile=256, overlap=32 at 4x on a 24GB card handles up to roughly 8K input without special casing.
  • The output tensor is the real memory hog, not the tiles: a 4K to 16K float32 RGB output is about 3GB. If you are memory-bound, write tiles to disk and stitch with a memory-mapped array instead of allocating the full output.
  • torch.inference_mode() over no_grad() is marginally faster and disables view tracking, which matters when you have many small tiles.

What this does not fix

  • Tiling does not recover global context. A model that cannot see the whole image cannot keep global lighting consistent; you will get subtle exposure drift across large tiles. Smaller overlap makes it worse, not better.
  • Overlap costs compute. overlap=32 on tile=256 means about 28 percent extra area processed. overlap=64 costs about 55 percent. That is the tradeoff.
  • Denoise has to come before this, not after. If you denoise per tile you re-introduce failure mode 1 through the back door.

If anyone has a better approach to the global-context problem (for example a low-res global pass to compute per-tile normalization targets), I would like to hear it. That is the part I have not solved cleanly.

Adding the one thing the post above skips: how to check whether tiling actually helped. Compare a centre crop at 100% against the same region of the pre-upscale image side by side, not the whole frame scaled down to fit a monitor. Downscaling for display hides exactly the artifacts tiling introduces. If you want a quick reference for what the failure looks like, there is a writeup at https://upgolab.com/en/blog/line-art-breaks-upscalers-in-a-way-photos-never-do that covers the line-art case specifically (where tiling damage is most visible).