Why Your AI Photos Are Worse than You Think
It took you twenty minutes to write the best prompt. Hit generate. The result? Promising–yet somehow mushy on the sides. The eyes are somewhat cloudy. That background texture? Mush. Sound familiar?
The thing that most people do not utter aloud is that much of the generated images by AI image generator have a resolution issue. Not due to the bad models. Since high-resolution generation is computationally costly, and most platforms silently render at 512×512 or 768×768 – resolutions that appear fine in a thumbnail, but collapse the moment you attempt to apply them anywhere beyond a thumbnail.
There AI upscaling comes in. And it is much more interesting than merely making the picture bigger.
The Difference between Upscaling and Stretching
The ancient methods of image resizing, bicubic or nearest-neighbor resizing, which Photoshop has been using over the years, is based on interpolating pixel values. The software essentially makes an educated guess as to the colour between two existing pixels and fills it in. What you have is a bigger picture but that is… blurrier. Such as a photocopy of a photocopy.
AI upscaling does not do this. It applies neural networks that have been trained on millions of image pairs: low resolution and high resolution to determine what detail should be present in a particular area. It’s not guessing between two known pixels. It’s hallucinating plausible texture based on context.
Imagine it in the following way: traditional upscaling poses the question what colour do we likely have here? AI upscaling poses the question: would this be a real picture, what do we have here?
The difference is crucial. An upscaled face using Real-ESRGAN or another similar model does not simply get larger, instead the skin pores open, eyelash strands are spaced apart, small catchlights are visible in the iris. The detail which was never present in the original is synthesised out of acquired visual knowledge. The wildest thing to think about.
The way the Models Work in Reality
Most modern upscalers are based on the architecture of a Generative Adversarial Network – a GAN – or more recently, a diffusion-based model. The GAN method opposes two networks; a generator that attempts to generate realistic high-res images, and a discriminator that attempts to detect hoaxes. They train until they both become sharper, until the generator is generating output that the discriminator can consistently fail to distinguish between it and a real photo.
The more recent diffusion-based upscalers are different. They introduce noise to the low-res image and then repeatedly remove noise to be steered towards increased detail. This is more likely to yield results with a more natural variation – less of that hyper-sharp slightly plastic quality of early GAN upscalers.
Both methods are not always superior. It will all be a matter of your source material and what you are doing with the output.
The face swap free Workflow Problem Nobody Talks About.
This is where things become really tricky. When you are doing portrait work, which is a large portion of AI image generation, upscaling and face enhancement are two distinct issues, which people always confuse.
generated images face swap free tools have rendered it trivially easy to transplant one face onto another body, which would be helpful until you realise that the composite would always have seam problems, lighting problems, and inconsistency of skin tones, which would be exacerbated rather than improved by standard upscaling. Upscaling enhances the existing contents of the picture, including artifacts.
The order is important: correct the compositing problems and face alignment before upscaling, not afterwards. Passing a face-swapped image through an upscaler will entrap those artifacts at four times the size.
Frequency and Detail: A Technical Detour Worth Taking
When engineers discuss image detail, they tend to refer to it as spatial frequency. Broads shapes and colour gradients, the fact that there is a face in the picture.
Low-frequency information is retained fairly well in traditional upscaling. It is totally ineffective in high-frequency reconstruction. AI upscaling is specifically good at synthesising plausible high-frequency detail.
But – and here is the point – the synthesised high-frequency detail is not the same thing as recovered detail. Assuming that there was none there to start with, the model is fabricating. In the majority of cases this appears convincing. It sometimes hallucinates illegible text, or puts wrinkles on a face that did not exist. The difference is significant when you are concerned about the accuracy and aesthetic quality.
Various Tools to Various Problems
Not every upscaling tool acts similarly, and the incorrect choice of this tool may result in a deep-fried look of your picture.
The most commonly used open-source is likely to be real-ESRGAN. It has been conditioned to degrade synthetic images – it has learned the appearance of compressed, noisy, blurry images and has been trained to undo those features. Very good with photos and photorealistic renders.
Topaz Gigapixel AI is the most popular commercial among photographers and digital artists. It has several models that are pre-configured to various kinds of content – faces, landscapes, styles of art. The outcomes may be truly breathtaking, but the cost shows that.
The built-in upscaler of Stable Diffusion (and other diffusion-based models) is intriguing in the sense that the original prompt can be used as guidance during upscaling. This implies it has the ability to add contextually relevant detail, so in case your prompt specified a rough stone wall, the upscaler can produce convincing stone texture as opposed to a generic texture pattern.
To any person operating more or less a platform based on an ai image generator, numerous of them now come with native upscaling features built into the interface. Quality is significantly different across platforms and it is worth trying your own tool on your own image against a standalone upscaler to determine which one can work better than the other in your application.
