Artificial intelligence (AI)-based image generators such as ChatGPT or Google Gemini often produce an almost perfect picture, but ask them to fix one detail and the model unexpectedly changes something unrelated. The reason lies in how modern AI models are built.
Most of them use a diffusion model in what is called latent space. This means that concepts such as "cat" or "dog" are compressed to key features and stored in compressed form so they can be quickly retrieved when creating an image. This approach saves computation but removes control: you cannot specify exactly which pixels need to be changed.
"Using latent space reduces computation, but you lose controllability because you are not adjusting individual pixels," explains Jiebo Luo, a professor of engineering at the University of Rochester. "As a result, you cannot easily control specific parts of the image."
Luo's lab, together with colleagues from NVIDIA (a developer of AI infrastructure), created one of the first models that works directly with pixels instead of latent space. The first version, PixelDiT, was presented at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) in 2026. The model makes it possible to focus on specific areas: for example, to fix only the pixels of glasses on a face or to change the color of a car.
The team then developed a more powerful model, PixelDiT2. It efficiently performs diffusion in pixel space and achieves almost the same performance as advanced models with latent space, while also giving control over individual pixels. A paper on PixelDiT2 will be presented in December at the Conference on Neural Information Processing Systems (NeurIPS), and its preprint is available on the arXiv server.
The researchers have now taken on an even more complex task: video generation. "Video is a big problem because you have to take care of smooth transitions from frame to frame, otherwise the human eye will notice unnaturalness," Luo says. "Besides, there is much more data." According to him, the advances of PixelDiT2 create a solid foundation for working with video in pixel space, which will give users new possibilities for controlling generated clips.
Note: for now these are laboratory models, not finished commercial products. No timeline has been announced for when such tools will become publicly available.