Skip to main content

Module 8 — Upscaling and face restoration

BoisClair's marketing agency asks for the walnut chair on a large poster — A2 at 300 dots per inch, i.e. 4961 by 7016 pixels. The 1024 by 1024 hero shot from module 3 is a tenth of that. This module is about going from a native-resolution generation to a print-ready image, and about restoring faces when the catalog includes a lifestyle shot with a person in it. Two families of tools, two failure modes to avoid.

Two families of upscalers​

Dedicated super-resolution networks — Real-ESRGAN, ESRGAN, SwinIR — take an image in and return a bigger image. They are fast (fractions of a second per megapixel on a consumer GPU), they preserve the original faithfully, and they cannot invent detail that was not implied by the original pixels. Real-ESRGAN with the x4plus model is the pragmatic default; x4plus_anime exists for illustrated content.

Diffusion upscalers — the SDXL upscaler, sd-x4-upscaler for SD 1.5, or the "hi-res fix" workflow — encode the input into latent space, add noise, and denoise at the target resolution with the U-Net. They can hallucinate plausible detail that was never in the original: skin pores, wood grain fibers, individual threads in a linen. They are slower and can drift the identity of the subject.

Which family to pick depends on whether you want a faithful enlargement (Real-ESRGAN) or a detail-enhanced enlargement (diffusion). For product photography, faithful is safer. For creative posters, hallucinated detail can look magnificent when it does not go wrong.

Real-ESRGAN: the pragmatic path​

from realesrgan import RealESRGANer
from basicsr.archs.rrdbnet_arch import RRDBNet
from PIL import Image
import numpy as np

model = RRDBNet(num_in_ch=3, num_out_ch=3, num_feat=64, num_block=23, num_grow_ch=32, scale=4)
upsampler = RealESRGANer(
    scale=4,
    model_path="RealESRGAN_x4plus.pth",
    model=model,
    half=True,
)

image = Image.open("boisclair_walnut.png")
np_img = np.array(image)
output, _ = upsampler.enhance(np_img, outscale=4)   # 4096 x 4096
Image.fromarray(output).save("boisclair_walnut_x4.png")

Three things worth knowing.

Fixed scale factors (typically x2, x4). Use outscale to reach an intermediate factor via internal resizing; the visible quality peak is at the trained factor.

Half precision. half=True cuts memory and roughly halves the time on a GPU with tensor cores. Rounding errors are invisible at print sizes.

Tile size on large inputs. Very large images run out of VRAM; RealESRGANer(tile=256, tile_pad=10) splits the image into tiles that fit and stitches them back. Padding between tiles avoids visible seams.

Diffusion upscaler with the "generate, then refine" pattern​

For hero shots on the BoisClair project, a two-step pattern beats a single-step upscale.

  1. Generate the hero at 1024 with SDXL base.
  2. Refine it with SDXL Refiner OR with the SDXL image-to-image pipeline at 1536 or 2048 with a low denoising strength (0.2 to 0.35). The pipeline adds plausible detail without drifting the composition.
  3. Optionally, run Real-ESRGAN x2 on top of the refined image to reach print resolution.
from diffusers import StableDiffusionXLImg2ImgPipeline

pipe = StableDiffusionXLImg2ImgPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16, variant="fp16",
).to("cuda")

hero = Image.open("boisclair_hero_1024.png").resize((2048, 2048))
refined = pipe(
    prompt="walnut mid-century dining chair, sharp focus, editorial catalog",
    image=hero,
    strength=0.25,
    num_inference_steps=25,
    guidance_scale=6.0,
).images[0]

The low strength is the whole trick. At 0.25 the model does not reinvent the chair — it only decides what the fine detail should look like at the new resolution.

Artifacts to watch​

Three failure modes recur across projects.

Waxy skin. Faces upscaled with a diffusion upscaler often look plasticised — pores are gone, edges too smooth. The fix is a dedicated face-restoration pass (below), not a stronger diffusion pass.

Cartoon lines on natural textures. Real-ESRGAN with the anime model on a photograph looks like a badly traced illustration. Match the model to the content type.

Hallucinated detail that reads as wrong. A diffusion upscaler can invent wood grain that flows against the direction of the plank, or fabric threads that do not weave together. Read the print at 100 % scale before shipping — the eye catches these at scale, not on a screen preview.

Face restoration​

For catalog shots featuring a person, both diffusion upscalers and Real-ESRGAN often mangle the face — one of the most difficult things for a super-resolution network to reproduce is a coherent identity. Two dedicated tools solve this cleanly.

GFPGAN and CodeFormer are face-specific networks trained to restore high-fidelity faces from degraded inputs. They detect faces in the image, restore each face separately, and blend the result back. CodeFormer has a w parameter that trades fidelity for quality (w=0.7 is a good default: recognizable identity, cleaned-up features).

from gfpgan import GFPGANer

restorer = GFPGANer(
    model_path="GFPGANv1.4.pth",
    upscale=2,
    arch="clean",
    channel_multiplier=2,
)
_, _, restored = restorer.enhance(np_img, has_aligned=False, only_center_face=False, paste_back=True)

The order of operations matters. Restore faces on the native-resolution image first, then upscale the whole image with Real-ESRGAN. The reverse order sometimes works but is more prone to identity drift on very small faces.

Three practical facts for the BoisClair poster.

Effective PPI at viewing distance. A poster viewed at two meters does not need 300 pixels per inch — 150 is fine. A brochure held at 30 centimeters needs the full 300. Print resolution is a distance-dependent number, not a universal target.

Save in the right format. 8-bit PNG for web, 16-bit TIFF for print at professional printers, JPEG at quality 95+ for social media. Never let a JPEG at quality 80 be your archive — recompression on the next edit accumulates.

Embed the ICC color profile. sRGB for screen, Adobe RGB or the printer's specific profile for print. Without a profile, the printer guesses, and the walnut turns orange.

The upscale ladder for BoisClair

Native SDXL 1024 → image-to-image at 2048 with strength=0.25 → Real-ESRGAN x2 to 4096 → optional face-restoration pass on any human present → export as 16-bit TIFF for the print run. Total time: about a minute on a consumer GPU, per hero image.

In summary​

  • Real-ESRGAN and its family produce faithful enlargements at high speed; diffusion upscalers hallucinate plausible detail but can drift subject and identity.
  • The reliable pattern for hero shots is native generation → image-to-image at higher resolution with low strength (0.25) → Real-ESRGAN x2 to reach print pixels.
  • Face restoration (GFPGAN, CodeFormer at w=0.7) is a dedicated step; run it before the final upscale, not after, to avoid identity drift on small faces.
  • Print resolution is viewing-distance dependent; embed the correct ICC profile and archive in a lossless format (16-bit TIFF or high-quality PNG).

Next module: keeping all this affordable — half precision, attention slicing, CPU offload, resolution versus memory, and time per image on modest hardware.