Most upscaling models are trained on 256-512px crops but people want to run them on 4K photos. The model OOMs, so you tile. The naive tiling loop has two failure modes that are easy to miss.
Failure mode 1: seam artifacts from independent normalization
If each tile is normalized separately (per-tile mean/std, or per-tile min/max), the tiles come back at slightly different brightness and you get a visible grid. Normalize over the whole image first, then tile. Obvious in hindsight, but it bites because the per-tile artifacts are subtle in a single tile and obvious in the composite.
Failure mode 2: edge tiles padded with zeros
Zero padding at the borders makes the model hallucinate dark edges. Reflection padding is better but you must crop it back correctly. Getting the crop math wrong by even one pixel produces a one-pixel dark line at every seam, which at 4x looks like a 4px band.
The pattern that works for me:
import torch
import torch.nn.functional as F
@torch.inference_mode()
def tiled_upscale(model, img, scale=4, tile=256, overlap=32):
# img: (1, C, H, W) float tensor, already normalized over the FULL image
_, C, H, W = img.shape
out = torch.zeros((1, C, H * scale, W * scale), dtype=img.dtype, device=img.device)
weight = torch.zeros_like(out)
step = tile - 2 * overlap
for y in range(0, H, step):
for x in range(0, W, step):
y0, x0 = max(0, y - overlap), max(0, x - overlap)
y1, x1 = min(H, y + tile + overlap), min(W, x + tile + overlap)
patch = img[:, :, y0:y1, x0:x1]
ph = (32 - patch.shape[2] % 32) % 32
pw = (32 - patch.shape[3] % 32) % 32
if ph or pw:
patch = F.pad(patch, (0, pw, 0, ph), mode='reflect')
up = model(patch)
up = up[:, :, : (y1 - y0) * scale, : (x1 - x0) * scale]
oy0, ox0 = y0 * scale, x0 * scale
oy1, ox1 = oy0 + up.shape[2], ox0 + up.shape[3]
out[:, :, oy0:oy1, ox0:ox1] += up
weight[:, :, oy0:oy1, ox0:ox1] += 1.0
return out / weight.clamp_min(1.0)
Averaging overlapping regions by weight (instead of last-write-wins) removes the remaining faint seams. weight.clamp_min(1) guards against holes if your step math ever leaves a gap.
Memory notes
- tile=256, overlap=32 at 4x on a 24GB card handles up to roughly 8K input without special casing.
- The output tensor is the real memory hog, not the tiles: a 4K to 16K float32 RGB output is about 3GB. If you are memory-bound, write tiles to disk and stitch with a memory-mapped array instead of allocating the full output.
- torch.inference_mode() over no_grad() is marginally faster and disables view tracking, which matters when you have many small tiles.
What this does not fix
- Tiling does not recover global context. A model that cannot see the whole image cannot keep global lighting consistent; you will get subtle exposure drift across large tiles. Smaller overlap makes it worse, not better.
- Overlap costs compute. overlap=32 on tile=256 means about 28 percent extra area processed. overlap=64 costs about 55 percent. That is the tradeoff.
- Denoise has to come before this, not after. If you denoise per tile you re-introduce failure mode 1 through the back door.
If anyone has a better approach to the global-context problem (for example a low-res global pass to compute per-tile normalization targets), I would like to hear it. That is the part I have not solved cleanly.