VFX Atlas
H 314 / 3212020s

Generative Inpainting

Generative inpainting. 2021–2022: LaMa research and more widely accessible generative image processing

Typeface · Manrope

DALL·E 2: documented inpainting demo example, 2022.
DALL·E 2: documented inpainting demo example, 2022.OpenAI / DALL·E 2, historical product example.View source ↗

Prompt

Apply Generative Inpainting to small distracting background imperfections, replacing them with seamless continuations of nearby surfaces. Match local color, material texture, grain, perspective, shadow softness, and edge sharpness so repaired regions have no visible patch boundaries or repeated patterns. Keep the original subject and composition.

Paste it into LetzAI with your own image to apply the effect ↗

Identify

Masked image areas1 are filled with newly synthesized content2.

Principle

A model fills in the gap based on the environment and, if necessary, text.

Example

LaMa, 2021; DALL·E 2, 2022.

Try it out

Replace an object; check shadows and continuous patterns.

Background

Inpainting fills in missing or selected areas within an image. Classical methods existed before generative AI; this slide focuses on machine learning-based augmentations from the early 2020s. LaMa was presented as a preprint in 2021 and at the WACV conference in 2022. The method uses Fourier convolutions for large-scale spatial context and also demonstrates periodic structures. DALL·E 2 made describable image changes with appropriate surroundings, shadows, and textures publicly visible in 2022. The illustration is a documented demo from that time, not a newly created historical reconstruction. The key learning question is which cues are preserved. A local exchange must match the perspective, light, and material of the surrounding image.

Distinction

A plausible addition does not necessarily reconstruct the original content that was actually concealed.

Aftermath

Inpainting works within the chosen image area. Unlike repair by copying, it can invent new objects and structures.

Further examples · 7 images

A couple of face-painted humans happily gazing into each other's eyes - crop 5 - inpainted (DSC03824)
02 A couple of face-painted humans happily gazing into each other's eyes - crop 5 - inpainted (DSC03824)Two people with face paint stand in a dense forest. Inpainting was used to reconstruct parts of their bodies and the foliage after cropping.Trougnouf with Stable Diffusion AI · CC BY 4.0 · Wikimedia Commons
Milite Ignoto in Altare della Patria in Vittoriano in Rome (Inpainting 2)
03 Milite Ignoto in Altare della Patria in Vittoriano in Rome (Inpainting 2)This image shows the Altare della Patria in Rome at night with a central statue removed. It demonstrates the removal of large foreground objects by synthesizing the background architecture.User:Codice1000.en and derivative work by Prométhée33 · Public domain · Wikimedia Commons
Watermark and inpainting on Aeroflot Airbus A321-211 VP-BUP Shagal
04 Watermark and inpainting on Aeroflot Airbus A321-211 VP-BUP ShagalA side-by-side view shows an Aeroflot aircraft with a watermark removed from the bottom left. The process fills the asphalt and runway lines to restore the clean original look.Alexander Mishin · CC BY-SA 3.0 · Wikimedia Commons
AI illustration: A user interface shows a red mask over a sofa to trigger a replacement
05 AI illustration: A user interface shows a red mask over a sofa to trigger a replacementA user interface shows a red mask over a sofa to trigger a replacement. The sidebar thumbnails demonstrate various generated armchair options that fit the room's perspective and lighting.Illustration generated for the Atlas (AI)
AI illustration: A comparison shows a power line tower removed from a forest landscape
06 AI illustration: A comparison shows a power line tower removed from a forest landscapeA comparison shows a power line tower removed from a forest landscape. The model synthesized matching tree patterns and textures to fill the void, creating a seamless natural background.Illustration generated for the Atlas (AI)
AI illustration: An editing interface shows a brush tool adding a trophy to a painting of
07 AI illustration: An editing interface shows a brush tool adding a trophy to a painting of An editing interface shows a brush tool adding a trophy to a painting of a dog. The generative model integrates the new object by matching the painterly style and lighting.Illustration generated for the Atlas (AI)

Question

Why can a seamlessly repaired photograph still be historically inaccurate?

Show possible answer +

The model generates a plausible addition from its patterns and the context; it does not know the actually missing content.

Sources