The fine art of computational photography and iOS



Software builds a better picture



HDR images are a sort of dynamic-range composite derived from multiple shots of an identically framed scene captured at very close intervals. Each shot is made with a different exposure duration (the brief time the shutter is open), and thus each captures a unique dynamic range: the span from light to dark and subtleties in between. An HDR algorithm takes the multiple shots, analyzes variances in dynamic range, and enhances the weakest areas (where highlights are blown out or shadow details are missing, for instance) to produce an often supernaturally exposed photo. It can sometimes be jarring, but HDR software can let you adjust that to be as natural or hyper-real as you wish. (See The best apps to create High Dynamic Range (HDR) photos.)

Photosynth shows the pieces of a panorama already captured, and frames the next segment as you move to help you align it into place.

HDR might seem at first like a form of bracketing, in which a photographer shoots a scene multiple times at slightly different exposure, apertures, focus, or flash. But bracketing is used to make sure you get one good shot. HDR relies on several shots that combine to make one enhanced photograph. (In fact, on most cameras, exposure bracketing captures the images needed for post-processing in HDR software.) HDR picture-taking appeared on the iPhone in iOS 4.1, although the smartphone can’t always snap pictures rapidly enough to make good HDR results. (We explained HDR on the iPhone at its introduction in 2010, and provided tips about how to use it best, too.)

Panoramic images, in contrast to HDR, work in the dimension of space, not time. To make a computed panorama, software captures a set of pictures across an area, so that simultaneity is less important than overlap. Some panorama software, whether built into a camera or a program you install on a desktop or mobile device, uses a literal approach, and simply joins pictures with distortion and overlay. But the best software performs analysis of intersections, and tries to remove skewing at the same time. You take a host of pictures, and out comes a landscape or a 360-degree view. These panoramas provide a richer visual description of an environment you’re in than a single shot, even with an extremely wide-angle lens, could ever do.

But HDR and panorama stitchers are two examples of what we’ll see an explosion of as developers start to put theories into practice, and see what Apple’s (and other makers’) powerful processors can do. There are already excellent apps available.

This panorama took a couple of minutes to assemble without any fussy tapping. Pan an iPhone around slowly, and the app provides guidance as to where it can grab the next photo segment.

Putting it to use

Dr. Levoy put his money (or, rather, time) where his mouth was by learning to program iOS apps, and releasing SynthCam, which is now free. The app relies on video recording (a frame every 1/30th of a second) rather than taking still images to create a shallow depth of field, an effect in which just a small part of an image is in focus, and objects closer and further away are blurred. This usually requires a single-lens reflex (SLR) camera to achieve. It’s a proof of concept, and thus a little tricky to use, but worth the effort. The app can also produce “tilt-focus” images, which make large objects appear like miniatures and models, and produce seemingly miraculous low-light photos.

The similarly named Photosynth (free), from Microsoft, is a panorama generator that emerged from a research project. It stitches together images into a three-dimensional view as you take photos. You can capture a real sense of the space you are in, and store it in a form of browsable 3D or create flat panoramic output as well. Several panoramic iOS apps offer different controls, although some perform simple overlaps, and don’t work very hard to create a seamless, corrected image. Panoramatic 360 ($1) is the best I tested for robust controls and options. It requires that you line up overlaps, which can produce quite good results, even if it’s more work than PhotoSynth.

There’s also the potential for more interesting camera technology to assist. The Lytro camera, released earlier in 2011, is the first one designed to capture a light field rather than static flat plane of light. The consumer-oriented camera has an odd form factor and takes low-resolution photos, but it can refocus an image after it’s taken. The secret is that the camera uses an array of micro-lenses interposed between a main lens and the CCD array that captures light. (No surprise that the company’s founder is a former graduate student of Dr. Levoy’s, and the professor serves on the company’s advisory board.)

Lytro Camera

Instead of capturing where the light falls, as in a conventional digital camera, the Lytro can calculate the direction of light rays, and trace them back through space to where they originate or reflect off an object. Thus, it can reset the focus (the theoretical focal plane) by recalculating where light rays would fall at different distances. There is no picture, as such; only a data that can be reconstructed into a picture. The Lytro is expensive ($399 base model), requires a deep lens array, and uses proprietary technology of all sorts, including a Flash-based Web-page viewer.

The future

Computational photography is a great fit with the camera on an iPhone, iPod touch, or iPad, because the relatively small size of the lens and image sensor requires more cleverness to obtain good or interesting photos. One could imagine a future iOS with an array of automatically employed techniques that users didn’t even have to set. Today, we flip an HDR switch to On, and use separate apps. Tomorrow, iOS could simply “magically” produce impossible photos that capture the world around us in a manner closer to what we see with our eyes.

[Glenn Fleishman, a Macworld senior contributor, worked at the Kodak Center for Creative Imaging in early 1990s, where he worked with the first commercially produced digital camera. His first SLR was a Canon AE-1.]

Products mentioned in this article

(2 items)

Subscribe to the Digital Photo Newsletter