The evolution of cosmic structures, the formation and growth of the first black holes and the connection to their baryonic environment are key unsolved problems in astrophysics. The X-ray Athena mission and the gravitational-wave Laser Interferometer Space Antenna (LISA) offer independent and complementary angles on these problems.
by Shane Colburn, Arka Majumdar, Alan Zhan
Conventional imaging systems comprise large and expensive optical components that successively mitigate aberrations. Metasurface optics offers a route to miniaturize imaging systems by replacing bulky components with flat and compact implementations. The diffractive nature of these devices, however, induces severe chromatic aberrations, and current multiwavelength and narrowband achromatic metasurfaces cannot support full visible spectrum imaging (400 to 700 nm). We combine principles of both computational imaging and metasurface optics to build a system with a single metalens of numerical aperture ~0.45, which generates in-focus images under white light illumination. Our metalens exhibits a spectrally invariant point spread function that enables computational reconstruction of captured images with a single digital filter. This work connects computational imaging and metasurface optics and demonstrates the capabilities of combining these disciplines by simultaneously reducing aberrations and downsizing imaging systems using simpler optics.
Shane Colburn, Arka Majumdar: Department of Electrical Engineering, University of Washington, Seattle, WA 98195, USA. Alan Zhan, Arka Majumdar: Department of Physics, University of Washington, Seattle, WA 98195, USA.
Modern cameras consist of systems of cascaded and bulky glass optics for imaging with minimal aberrations. Although these systems provide high-quality images, the improved functionality comes at the cost of increased size and weight, limiting their use for a variety of applications requiring compact image sensors. One route to reduce a system’s complexity is via computational imaging, in which much of the aberration correction and functionality of the optical hardware is shifted to postprocessing in the software realm, enabling high-quality images with significantly simpler optics. Alternatively, the designer could miniaturize the optics by replacing them with diffractive optical elements (DOEs), which mimic the functionality of refractive systems in a more compact form factor. Metasurfaces are an extreme example of such DOEs, in which quasiperiodic arrays of resonant subwavelength optical antennas impart spatially varying changes on a wavefront. These elements are of wavelength-scale thickness, enabling highly compact systems, while the large number of degrees of freedom in designing the subwavelength resonators has enabled unprecedented functionalities and flat implementations of lenses, holographic plates, blazed gratings , and polarization optics.
Although separately both computational imaging and metasurfaces are promising avenues toward simplifying optical systems, a synergistic combination of these fields can further enhance system performance and facilitate advanced capabilities, for example, use in full visible spectrum imaging with metasurfaces. Designing achromatic metasurface lenses for imaging under broadband illumination remains an outstanding problem in the metasurface community. The strong chromatic aberrations in metasurfaces originate from both the local resonant behavior of the subwavelength optical scatterers and the phase wrapping discontinuities arising from the spatial arrangement of the scatterers. For lenses, this chromaticity manifests as wavelength-dependent blur in images, which constrains metasurface-based imaging to narrowband operation. There is a vast body of work attempting to solve this problem; however, thus far, the presented solutions work either for discrete wavelengths or for narrow bandwidth. Here, we design the optical hardware in conjunction with computational postprocessing to realize a full-color imaging system comprising a single metasurface and a computationally inexpensive digital filter that can generate high-quality images under broadband white light illumination spanning the whole visible regime.
The three-dimensional (3D) point spread function (PSF) of a linear, shift-invariant optical system fully characterizes its behavior. At the image plane of an optical system, the 2D PSF corresponds to an image of a point source with size and shape related to the system’s geometry and aberrations. As the wavelength changes, the image plane shifts due to chromatic aberrations, inducing color-dependent blur in captured images because of the fixed location of the image sensors. For metasurface optics, this focal shift is inversely proportional to the optical wavelength, severely blurring polychromatic images. We mitigate this blur by engineering a metasurface with a PSF, which is invariant across the whole visible regime. This technique has been explored previously with macroscopic refractive optical systems, in which the optical wavefront is coded using a phase mask to provide an extended depth of focus (EDOF). This EDOF makes the system tolerant to focal shifts due to the preservation of spatial frequency information across the depth of the smeared-out focal spot. This comes with the tradeoff of reduced signal-to-noise ratio (SNR) and blurring of the captured image because incident light is spread over a greater volume; however, unlike a simple lens that has wavelength-dependent blur, an EDOF system can have spectrally invariant blur over a wide frequency band, the bandwidth of which increases with the depth of focus (see the Supplementary Materials for a performance comparison of metasurface systems with different depths of focus). The spectral invariance of the induced blur enables postprocessing with a single wavelength-independent filter to retrieve a high-quality image.
The requirement of a secondary phase mask for wavefront coding increases the system’s size and complexity. Furthermore, these phase masks are often freeform in nature (that is, characterized by rotational asymmetry or higher-order polynomials) and are challenging to fabricate by traditional means such as diamond turning and multistage lithography for making diffractive elements. With the flat nature of metasurface-based systems, however, we can convert a freeform element to a compact and uniform-thickness device using a single lithography stage. This design freedom also enables combining the lensing and wavefront coding functionalities into a single element. A variety of different wavefront coding masks can produce an EDOF. Typically, these masks will produce nondiffracting beams over a wide range and, in conjunction with a lens, can generate an elongated focus. Cubic functions are a common choice because of their simplicity and rectangular separability. Furthermore, it can be shown via the stationary phase method applied to an ambiguity function representing an optical transfer function that, for a phase mask, which is a monomial, the modulation transfer function (MTF) is insensitive to misfocus (for example, chromatic focal shift) if and only if the mask function is cubic. Owing to this wavelength insensitivity and their widespread usage in extended depth of field systems, we have elected to use a cubic phase term in this work as well. Here, we design a single element capable of simultaneously focusing light and coding the wavefront to increase the depth of focus with phase of the form below
where x and y are the in-plane coordinates, λ is the operating wavelength, f is the nominal focal length, L is half of the aperture width, and α signifies the extent of the cubic phase. We designed two metasurfaces: one with α = 0, which is a simple lens, and one with α = 55π, which has an EDOF. Both devices had a nominal focal length of 200 μm at 550 nm. Our devices consist of cylindrical silicon nitride nanoposts of thickness 633 nm on top of a silicon dioxide substrate (Fig. 1, A and B) positioned on a square lattice with a period of 443 nm. Our choice of nanoposts, as opposed to nanofins or V-shaped antennas, enables polarization-independent behavior, and the high bandgap of silicon nitride enables transparent operation and high efficiency across the visible band. Each nanopost mimics a truncated waveguide with low-reflectivity top and bottom interfaces that induce low-quality factor resonances. Incident light couples into modes supported by the nanoposts, which then shift the phase of the light before coupling the light to a transmitted free-space mode. By adjusting the diameter, the modal structure supported by the nanopost can vary, changing the modes’ ensemble behavior and inducing different phase shifts. Because of our interest in broadband visible regime operation (400 to 700 nm), we selected a central nominal design wavelength of 550 nm for our nanoposts. We simulated the transmission coefficient as a function of post diameter via rigorous coupled-wave analysis (RCWA) (Fig. 1C). The transmission coefficient exhibits more than a 2π change in phase and uniform amplitude over a wide range. In designing our metasurfaces, we used a set of 10 different phase steps between 0 and 2π and avoided drops in amplitude by selecting diameters off the resonance dips. We simulated both metasurfaces to analyze their performance as a function of wavelength. Figure 1 (D and E) shows the chromatic focal shift for the α = 0 and α = 55π designs, respectively, in which the dashed black lines indicate the desired focal plane. We see that although only green light (550 nm) is in focus for α = 0, all simulated wavelengths impinge on the desired plane as part of an EDOF for α = 55π. We fabricated the metasurfaces using electron-beam lithography and dry etching. Figure 1 (F and G) shows optical images of the final devices (see the Supplementary Materials for scanning electron micrographs of the nanoposts). The fabricated α = 0 and α = 55π lenses demonstrate average measured focusing efficiencies of 63 and 57%, respectively (see the Supplementary Materials), for the wavelengths tested, comparable to existing visible wavelength metalenses. Hereafter, we denote the α = 0 metasurface as the singlet metalens and the α = 55π metasurface as the EDOF metalens.
(A) The metasurfaces are made up of silicon nitride nanoposts, where the thickness T, lattice constant p, and diameter d are the design parameters. (B) Schematic of a metasurface comprising an array of nanoposts. (C) Simulation of the nanoposts’ transmission amplitude and phase via RCWA. Simulated intensity along the optical axis of the singlet metasurface lens (D) and EDOF metasurface (E), where, going from top to bottom in each panel, 400, 550, and 700 nm wavelengths are used. The dashed lines indicate the desired focal plane where the sensor will be placed. Optical images of the singlet metasurface lens (F) and the EDOF device (G). Scale bars, 25 μm.
Our computational imaging system poses a problem of the matrix form f = Kx + n , where the desired image x has been blurred by the system kernel K and corrupted by noise n to produce the captured image f. A variety of different methods are available for estimating x, such as the linear Wiener filter or regularized optimization-based approaches. For this work, we chose the Wiener filter due to its low computational complexity. Moreover, the reconstructed image quality is comparable to that of more advanced deconvolution methods (see the Supplementary Materials for performance comparisons with total variation-regularized deconvolution). We obtain the kernel K, as required for filtering the image, by a calibration PSF measurement. We measured the PSFs and calculated the MTFs for both the singlet and EDOF metalenses for red (625 nm), green (530 nm), and blue (455 nm) light-emitting diode (LED) illumination (Fig. 2) (see the Supplementary Materials for further information on the MTF measurement and comparison with theory and illumination with different bandwidth sources). For the singlet, the tightly focused spot with green illumination differs drastically from the large blurs under red and blue illumination, translating to the strong distinction in the MTFs under these illumination conditions. With this design, the zeros in the MTFs for the red and blue wavelengths result in an unrecoverable loss of spatial frequency information and preclude computational reconstruction. For the EDOF metasurface, however, not only are the MTFs wavelength-invariant but they also have no zeros until the cutoff spatial frequency. The preservation of this spatial frequency content enables computational reconstruction to retrieve the desired image x.
The PSFs of the singlet metalens (top row) and EDOF lens (bottom row) were measured under blue (A and E), green (B and F), and red (Cand G) illumination conditions. Scale bars, 25 μm. The MTFs were also calculated for both designs (D and H). In both (D) and (H), a normalized frequency of 1 corresponds to the same cutoff frequency of 579 cycles/mm.
To demonstrate the imaging capability of the system, we illuminated patterns on standard printer paper at object distances much greater than the focal length, on the order of a few centimeters. The metasurfaces would then form images by focusing down the scattered light. We first examined narrowband (~30-nm bandwidth) imaging performance under separate red (625 nm), green (530 nm), and blue (455 nm) illumination with LEDs. Figure 3 (C to E) compares the captured images of a portion of the 1951 Air Force resolution chart (Fig. 3A) by the singlet lens to those by the EDOF lens with and without deconvolution. Although the pattern is in focus for green light, upon switching to blue or red illumination the images captured via the singlet undergo severe distortion. For the EDOF lens without deconvolution, the image is blurry at all wavelengths, but the blur appears uniform across wavelengths. With postcapture deconvolution, however, the resulting images appear in focus for all wavelengths, providing a substantial performance improvement with greatly reduced chromatic aberrations relative to those of the singlet. We quantified this performance improvement in terms of the structural similarity (SSIM) of the Air Force pattern images. The deconvolved EDOF system’s images had a SSIM 0.209 higher compared to those of the α = 0 system (see the Supplementary Materials for further details of this calculation). We also imaged more complex patterns, including a black and white binary Mona Lisa pattern (Fig. 3, B and F to H), and also observed mitigation of the chromatic aberrations using the EDOF lens. Although the SNR of our deconvolved images is lower compared to that of the in-focus green images with the singlet metalens, our system exhibits in-focus images over a broad wavelength range. In conjunction with computational algorithms optimized for low-light imaging scenarios comparable to our experimental setup, imaging performance can be further improved.
The appropriately cropped original object patterns used for imaging are shown in (A) and (B). Images were captured of the 1951 Air Force resolution chart with the singlet metalens (C) and the EDOF lens without (D) and with deconvolution (E). Images were also taken of a binary Mona Lisa pattern with the singlet metalens (F) and the EDOF device without (G) and with deconvolution (H). Scale bars, 20 μm.
Finally, we tested the system under broadband illumination using a white light source (Fig. 4). Under this condition, the singlet lens significantly blurred color-printed RGB text (Fig. 4A), with noticeably better image quality for the green letter G as the lens was positioned to be in focus for green light. Figure 4A also shows the spectrally uniform blurring of the RGB text formed by capturing the same object pattern with the EDOF lens. After deconvolution, we can make out each individual character, whereas the blue B is blurry and the red R is unintelligible in the image captured directly with the singlet lens. The deconvolved images do exhibit some erroneous horizontal and vertical lines arising from the asymmetric shape of the PSF, which can produce directional artifacts, but these can be corrected by more advanced deconvolution as well as by using a rotationally symmetric PSF. Figure 4B demonstrates a similar image quality improvement for ROYGBIV text, where the characters are substantially blurred by the singlet metalens but are in focus after capturing with the EDOF device and deconvolution. For the case of a rainbow pattern (Fig. 4C), the chromatic blur induced by the singlet obscures individual color bands and the green stripe is barely evident, whereas the deconvolved EDOF image shows separate bands and edges. For a landscape image (Fig. 4D) with multicolor flowers and leaves, stem and leaf structures severely blurred by the singlet are in focus, and color ringing artifacts in the blossoms are reduced, in the deconvolved EDOF image.
Images were taken under white light illumination of color printed RGB (A) and ROYGBIV (B) text, a colored rainbow pattern (C), and picture of a landscape (D) with a blue sky, green leaves, and multicolor flowers. The appropriately cropped original object patterns used for imaging are shown in the left column. Scale bars, 20 μm.
Compared to metasurface designs where the achromatic bandwidth is narrow or discrete wavelength images are superposed to produce a color image, to the best of our knowledge, we are the first to demonstrate in-focus full-visible-spectrum direct imaging with white light. We observe this behavior in Fig. 4, where when a single image is captured, we have not only red, green, and blue light in focus after deconvolution but also intermediate colors, such as yellow, orange, and violet. Furthermore, our metalenses rely on simple transmissive scatterers that could be extended to use a variety of different geometries and materials, whereas existing broadband achromatic metalenses require scatterers with carefully designed dispersion characteristics. With our system, however, although the digital filter in conjunction with our modified phase mask enables broadband color imaging, the required postprocessing complicates the system and introduces a delay time to deconvolve the captured image. For many photography and video applications, this would not pose an issue as any captured frames could be saved and then deconvolved offline. For real-time imaging, our system would also work as our filter relies on the O(N log N) fast Fourier transform algorithm, which can be accelerated via field programmable gate arrays (FPGAs) or graphics processing units (GPUs). These hardware acceleration techniques would require additional circuitry and increase system complexity, requiring the designer to balance the system’s requirements and costs for the given application. Our implemented system also suffers from a limited space-bandwidth product arising from our small (200 μm) focal length and aperture width. Typically, the space-bandwidth product of an optical system will decrease as the system dimensions are scaled down, reducing the information capacity and the number of resolvable points in an image. The small aperture of our metalenses also limits light collection, which reduces the SNR and necessitates higher incident power or increased exposure time. These limitations, however, are not inherent to our hybrid optical-digital system, but instead they arise from our short focal length and would be present in any implementation at the same length scale.
The reported system combines computational imaging and an EDOF metasurface with a numerical aperture ~0.45 to image full visible spectrum object patterns with minimal chromatic aberrations, making our devices well suited for microscopy, hyperspectral imaging, and ultrathin cameras. To the best of our knowledge, we have also imaged with the shortest focal length metasurface to date with more than a 2.4× reduction in the focal length compared to the shortest value previously. Although this system must contend with geometric aberrations, these can be circumvented by further co-optimization of the optical element [for example, by stacking metasurfaces] and postprocessing algorithms. In combining optical metasurfaces and computational imaging, this work provides a model for designing hybrid systems where the optical hardware and software together generate high-quality images while minimizing system size and complexity.
The nanoposts were designed to give 0 to 2π phase for wavelengths spanning the entire visible regime (400 to 700 nm) as the diameter of the posts was varied. Figure S1 shows the transmission amplitude and phase for three representative wavelengths at 400, 550, and 700 nm, calculated via RCWA simulation. The refractive indices used for both the silicon nitride posts and the silicon dioxide substrate accounted for dispersion. To ensure that our nanopost design provided weakly coupled pillars that would allow us to make use of the unit cell approximation in implementing our phase profiles, we simulated the transmission amplitude (fig. S2A) and phase (fig. S2B) as a function of diameter as we swept the lattice constant to show minimal change in phase over a wide range of lattice constants. To design the metasurfaces nominally for 550 nm, the RCWA data served as a lookup table for mapping the phase at a given position to the post diameter which most accurately provides the desired phase. Because of the large spatial extent of our designs, the memory requirements were beyond our available computational resources to perform a full finite-difference time-domain simulation of our devices. Hence, we used the RCWA-calculated transmission coefficients and modeled our metasurfaces as complex amplitude masks and simulated their performance by evaluating the Rayleigh-Sommerfeld diffraction integral using an angular spectrum propagator.
A 633-nm film of silicon nitride was first deposited on a fused silica wafer substrate via plasma-enhanced chemical vapor deposition. The wafer was temporarily coated with a protective photoresist layer and diced into smaller pieces before cleaning by sonication in acetone and isopropyl alcohol. The sample was then spin-coated with ZEP-520A before sputtering 8 nm of Au/Pd as a charge dissipation layer. The sample was exposed using a JEOL JBX6300FS electron beam lithography system, and the charge dissipation layer was removed by type TFA gold etchant. After developing in amyl acetate, a layer of aluminum was evaporated onto the sample, and after performing lift-off, an aluminum hard mask was left on the silicon nitride layer for subsequent etching. The sample was etched using an inductively coupled plasma etcher with a CHF3 and O2chemistry, and the remaining aluminum was removed by immersing in AD-10 photoresist developer. Scanning electron micrographs of the fabricated devices are presented in fig. S3.
The focal planes of the fabricated metasurfaces were characterized via the experimental setup presented in fig. S4. Light from a fiber-coupled LED illuminated the metasurface under test, and a custom microscope assembled from a translatable stage, objective, tube lens, and camera took snapshots of the focal plane of the device. To measure the efficiency (fig. S10), the same setup was used with the addition of a flip mirror, pinhole, and photodetector (Newport 818-SL). The efficiency was calculated by taking the ratio of the power at the focal plane to that of the incident beam. The incident beam power was found by measuring the power through a piece of glass with the pinhole aperture set to image a region equal to the width of the metalens. The camera was corrected for dark noise by taking a sequence of calibration images with the lens cap on. We determined the MTFs of our lenses by Fourier transforming and then taking the magnitude of the measured focal spot.
Images were captured using the setup shown in fig. S5. Light from a fiber-coupled LED is incident off-axis on a pattern printed on standard 8.5 × 11 paper. The metasurface created an image near its focal plane by focusing light scattered off the printed pattern, and a translatable microscope consisting of an objective, tube lens, and camera captured this image. Before image capture, the camera’s dark noise was subtracted after taking a sequence of pictures with the lens cap on.
section 1. Effect of the cubic phase strength α on image quality
section 2. Image comparison for different deconvolution methods
section 3. Assessing chromatic invariance by SSIM of images
section 4. Comparison of theoretical and experimental MTFs with nonzero source bandwidth
section 5. Off-axis metalens performance
fig. S1. Transmission amplitude and phase of the nanoposts as a function of diameter.
fig. S2. Transmission amplitude and phase of the nanoposts as a function of lattice constant and diameter.
fig. S3. Scanning electron micrographs of the fabricated metasurfaces.
fig. S4. Experimental setup for characterizing the focal plane.
fig. S5. Experimental setup for imaging with metasurfaces.
fig. S6. Images captured with systems of different cubic phase strength.
fig. S7. Image quality comparison for different deconvolution methods.
fig. S8. Comparison of theoretical and experimental MTFs with nonzero source bandwidth.
fig. S9. Simulated off-axis performance of the singlet and EDOF metalenses.
fig. S10. Efficiencies of the singlet and EDOF metalenses.
This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial license, which permits use, distribution, and reproduction in any medium, so long as the resultant use is not for commercial advantage and provided the original work is properly cited.
Copyright © 2024 Creative Commons Papers International Association | Brussels (BE) | Editorial staff: info(at)tech-cc.eu
Leave a Reply