Light field photography, microscopy, and illumination

Marc Levoy, Computer Science Department, Stanford University

The light field is a four-dimensional function representing radiance along rays as a function of position and direction in space.  At Stanford we have built a number of devices for capturing light fields, including (1) an array of 128 synchronized video cameras, (2) a handheld camera in which a microlens array has been inserted between the main lens and sensor plane, and (3) a microscope in which a similar microlens array has been inserted at the intermediate image plane.

The third device permits us to capture light fields of microscopic biological (or man-made) objects in a single snapshot.  From these light fields, we can generate perspective flyarounds using light field rendering, or 3D focal stacks using digital refocusing.  Applying 3D deconvolution to these focal stacks, we can produce a set of cross sections, which can be visualized using volume rendering.  By inserting a second microlens array and video projector into the microscope's illumination path, one can control the light field falling on a specimen, as well as record the light field leaving it.

In this talk I will describe a prototype system we have built that implements these ideas, and I will demonstrate three applications for it: microscope scatterometry - measuring reflectance as a function of incident and reflected angle, "designer illumination" - illuminating one part of a microscopic object while avoiding illuminating another, and correcting optical aberrations digitally - using the illumination system as a "guide star" and the recording system as a Shack-Hartmann sensor.  Time permitting, I will also describe some ongoing work on super-resolution reconstruction of microscope light fields.