Kaili Cao 曹开力

Images · Image Reconstruction

Image Reconstruction

Reconstructing well-sampled images with a controlled point spread function from a telescope's undersampled exposures, for the Nancy Grace Roman Space Telescope and beyond.

In brief

A telescope never records the sky exactly. Even a perfect point of light, such as a distant star, is spread into a small blur, called the point spread function (PSF), and the camera then records that blur in square pixels. This page is about turning raw exposures into images whose blur is known precisely, because precise measurements depend on it.

The Nancy Grace Roman Space Telescope, launched in August 2026, observes from space, so its PSF is sharp and stable, limited mainly by diffraction. But its pixels, 0.11 arcseconds across, are about as wide as the PSF itself. To record a pattern faithfully, you need more than two samples across its finest detail (the Nyquist–Shannon sampling theorem); for Roman, that would mean pixels about half as wide as the PSF. So a single Roman exposure is undersampled: details smaller than a pixel are lost or scrambled, as in the left panel of the figure below.

Why does that matter? Roman's main cosmology survey uses weak gravitational lensing: matter between us and distant galaxies bends their light and stretches their images by about 1%. But galaxies are not round to begin with: their own shapes are elongated by about 30% on average, in random directions. So the signal appears only when the shapes of millions of galaxies are averaged, and any distortion added by the telescope or by the processing must be known to much better than 1%.

The fix starts with dithering: Roman shifts slightly between exposures, so that different exposures sample the sky at different positions within a pixel (the right panel). Image reconstruction then combines the exposures into one well-sampled image. Each output pixel is a weighted sum of nearby input pixels. Because the weights decide how light from neighboring positions is mixed, they also set how blurred the output image is: its PSF. My work is on choosing the weights so that the output PSF matches a chosen target as closely as possible, while keeping the noise low (weights that sharpen an image by subtracting neighboring pixels also amplify its noise), and on computing them fast enough for a survey of billions of pixels.

One exposure-202position (pixels)
Two dithered exposures-202position (pixels)
  • true star profile
  • samples, exposure 1
  • samples, exposure 2 (shifted by half a pixel)
Illustration (one-dimensional, not real data). Left: a star sampled by pixels about as wide as its point spread function loses its shape. Right: a second exposure shifted by half a pixel ("dithering") fills in the gaps; image reconstruction combines such exposures into one well-sampled image. Made with AI assistance (Claude).

New to this topic? The Glossary explains the terms in plain language; underlined terms link to it.

In more depth

The Nancy Grace Roman Space Telescope, launched on August 30, 2026, observes from space, where its point spread function (PSF) is stable and sharp, limited mainly by diffraction. That is both a blessing and a curse. Roman's pixels, 0.11 arcseconds across, are about as large as the PSF itself, but fully sampling an image would need pixels about half that size. So native Roman images are undersampled, and the best current methods for measuring galaxy shapes need well-sampled images.

Shapes matter because the imaging part of Roman's High-Latitude Wide-Area Survey, the High Latitude Imaging Survey, is one of the main cosmology surveys, and weak gravitational lensing is one of its main science drivers. Matter in the foreground distorts the images of distant galaxies, but only at the percent level, while the intrinsic shapes of galaxies differ by about 30%. To recover the lost detail, Roman shifts slightly between exposures (dithering), and the exposures are later combined into well-sampled images. Each output pixel is a weighted sum of input pixels, and the choice of weights also decides the output PSF. In simulations of Roman's survey pattern, Drizzle, the traditional method for space telescopes, turns the image of a star into a hexagon (Hirata et al. 2024, Figure 9). IMCOM chooses the weights so that the output PSF is as close as possible to a target that the user specifies, which is what weak lensing needs.

I think of this software as a scientific instrument: how it is designed, tested and calibrated decides what the data can tell us, as much as the telescope itself does.

For statisticians: reconstruction as a linear inverse problem

In statistical terms, this is a linear inverse problem. If the input pixels are x = Af + ε, where f is the sky, A encodes the PSFs and sampling, and ε is noise with covariance Σ, every linear method, Drizzle, IMCOM and Effortless alike, produces y = Wx, which should approximate a target Tf with a well-defined PSF. The weights trade leakage, a bias (WA − T)f that depends on the sky, against noise, a variance with covariance WΣWT; IMCOM, for example, minimizes ‖WA − T‖2 + κ tr(WΣWT). In my one-dimensional study (Cao and Roman HLIS Cosmology PIT 2026), I framed a PSF as the probability distribution of where a photon from a known direction lands, and showed that regridding and coaddition change how information is represented but cannot add to it. Effortless computes its weights in Fourier space instead of solving IMCOM's large linear systems, which is where its speed comes from.

What I have done

I led the refactoring of IMCOM into PyIMCOM: object-oriented code with improved data structures and new linear algebra strategies, which reduced the computing cost by about an order of magnitude (Cao et al. 2025). I then tuned its hyperparameters, comparing 26 configurations in each of five bands on 12 criteria; wider Gaussian target PSFs, for example, led to better images (Cao et al. 2026). PyIMCOM produced the IMCOM coadds of OpenUniverse2024, a shared simulated sky for Roman and Rubin, and it is the platform for several team papers (see Image Analysis).

Six image panels: simulated Roman sky images (top row) and injected stars (bottom row), each coadded by PyIMCOM's three linear algebra kernels.
Simulated Roman sky images (top) and injected stars (bottom), coadded by PyIMCOM's three linear algebra kernels (columns: Cholesky, iterative, empirical). Top half of Figure 5 of Cao et al. 2025, CC BY 4.0.

Even so, IMCOM is expensive: reconstructing one native Roman image (one detector's exposure) takes about a day on a single CPU core. That is affordable for precision cosmology, but it makes IMCOM less attractive for other uses. A study in one dimension (Cao and Roman HLIS Cosmology PIT 2026) led me to a new algorithm for linear image reconstruction, first called Fast IMCOM. Since it does not necessarily combine images, I renamed it Effortless (EFFicient Optimal image ReconsTruction using LESS memory). Like IMCOM, it controls the PSFs of output images, but by avoiding IMCOM's most laborious calculations, it needs 50 to 60 times less computing time (Cao 2026a): in simulations of Roman's Medium and Wide Tiers, with the same inputs and target PSFs as earlier tests of PyIMCOM, it takes about 2×102 core hours per filter per square degree of sky, against PyIMCOM's 1.1×104, so a native Roman image takes about half an hour instead of a day. It also has a smaller memory footprint. The idea is to treat the weights given to input pixels as samples of an underlying weight field, and to design that field directly.

I describe Effortless in two companion papers. In one, I present its mathematical formalism, software implementation and practical issues (Cao 2026b). Foremost, while the Nyquist–Shannon sampling theorem remains true, its conditions differ subtly but importantly from the problem in survey data processing, and the effects of finite sampling can be greatly reduced by a simple calibration after measurement. Imperfections from numerical artifacts, from the finite windows of input pixels, and from input pixels that are unavailable are understood and under control. In designing the software, I aimed for a clear object-oriented structure, an intuitive data model and a friendly interface, and its programming interface is general enough for uses beyond weak lensing and beyond Roman. The code is public on GitHub.

In the other, I apply Effortless to simulated Roman images (Cao 2026a). It makes PSFs uniform and regular across reconstructed images more efficiently than IMCOM, and it has the potential to control systematic errors better. For ideal point sources, a single image reconstructed by Effortless, combined with the calibration above, can give measurements comparable to or better than those from a set of about six images coadded by IMCOM. Whether the calibration carries over to noisy galaxies, which is what weak lensing measures, is not yet clear. And although both algorithms were designed for weak lensing, Effortless can serve studies of both static features and dynamic aspects of the Universe: because it puts images taken at different times onto the same pixel grid and PSF, it can help time-domain astronomers compare them. Its efficiency and interpretability also open new possibilities for reducing measurement errors further.

Four image panels: simulated sky images (top) and injected stars (bottom), coadded by PyIMCOM (left) and reconstructed by Effortless (right).
The same simulated Roman sky images (top) and injected stars (bottom), coadded by PyIMCOM (left) and reconstructed by Effortless (right). First two rows and columns of Figure 1 of Cao 2026a.

Like IMCOM, Effortless assumes that the astrometry, flux calibration and PSF models it is given are correct. In reality, all of them have errors, which is one reason for the work ahead.

Papers

First-author papers

Coauthored papers

What comes next

Effortless, an idea of my own from my PhD, is the first to have started to flourish. Next come its applications: images for Roman's weak lensing survey, reference images fast enough for Roman's prompt alerts, deep images from its time-domain surveys, and joint analyses with Rubin's images. Then come a more general understanding of PSF residuals, and models of the noise covariance in reconstructed images.

How this fits into my next five years: Research Directions.

Terms on this page (15)
Point spread function (PSF)
The image that a telescope and camera make of a single point of light, such as a distant star: a small blur. Every image is the true sky blurred by the PSF, so knowing the PSF precisely is essential for measuring shapes.
Pixel
One element of a detector or digital image. A pixel value is best thought of as a sample of the sky's brightness at one point, after blurring by the telescope and by the way the pixel collects light across its area, not as a little square of sky.
Diffraction limit
The sharpest image a telescope can make, set by the wave nature of light and the size of its mirror: even a perfect telescope spreads a point of light into a small pattern. Space telescopes, free of the blurring by Earth's atmosphere, can reach it.
Arcsecond
A unit of angle on the sky: 1/3600 of a degree. The full Moon is about 1,800 arcseconds across; Roman's pixels are 0.11 arcseconds across.
Nyquist–Shannon sampling theorem
A result from signal processing. Any signal, including an image of a star, can be written as a sum of waves of different sizes (a Fourier series). If the smallest wave in it has a certain size, the signal can be recovered exactly from samples taken more than twice per wavelength of that smallest wave. Sample more coarsely, and fine detail is lost or disguised as coarser patterns, which is called aliasing.
Undersampled, well-sampled
An image is undersampled when its pixels are too coarse to capture its finest details (fewer than two samples across them), and well-sampled when they are fine enough. Roman's single exposures are undersampled; reconstructed images are well-sampled.
Weak gravitational lensing
The slight bending of light from distant galaxies by the gravity of matter in front of them, which stretches their images by about 1%. Averaged over millions of galaxies, it maps all the matter, including dark matter, and so tests cosmological models.
Dithering
Shifting the telescope slightly between exposures, often by fractions of a pixel, so that different exposures sample the sky at different positions. Combined, they recover detail that a single undersampled exposure misses.
Inverse problem, linear inverse problem
Inferring the causes from their observed effects, such as the true sky from blurred, sampled and noisy images. It is linear when the observations are a weighted sum of the unknowns plus noise, as in image reconstruction.
PSF leakage
How far a reconstructed image's PSF falls short of its target: the part of the result that is a bias, depending on the sky itself, rather than noise.
IMCOM
Short for IMage COMbination: an image-combination method that chooses the weights of input pixels so that the output image has a PSF as close as possible to a target the user specifies, while keeping the noise under control. PyIMCOM is its modern implementation for Roman, and Effortless is a faster successor.
Coadd, coaddition
An image made by combining several exposures of the same part of the sky, which reduces noise and, with dithering, recovers detail lost to undersampling.
Systematics, systematic errors
Errors that push results in a consistent direction, instead of scattering them randomly. More data do not average them away, so they must be understood and corrected. Uncorrected noise patterns, such as stripes, can become systematics when they mimic the signal.
Astrometry
Measuring the precise positions of objects on the sky. Combining images needs accurate astrometry, so that each pixel is placed at the right spot.
Flux calibration
Converting the numbers a detector records into physical brightness (flux: the energy arriving per second per unit area), so that measurements from different exposures and instruments agree.

Full glossary