SIGHTINGS
Needlework
Felice Frankel
One of the most rewarding parts of my work is making
connections—first by introducing researchers from
disparate fields whose work appears (to me) to have visual
commonalities. Other times it is methodologies that share a
common thread.
When I visited Michael Cohen, a senior researcher at Microsoft
Research, some time ago, he showed me several of their consumer
applications. One application being developed as part of the
Microsoft Expression designer toolkit under the code name
Acrylic includes a "stitching" feature called
PhotoMontage, developed to enable amateur photographers to
seamlessly and easily stitch together several images (of the
Grand Canyon, for example) to create a panoramic view.
Coincidentally, a week before the visit, I had a conversation
with John Hart, a graduate student in mechanical engineering at
MIT. We discussed the problem of using a scanning electron
microscope for samples larger than the instrument's field of
view. SEMs, unlike optical microscopes, create images with
amazing depth of field, surface contrast and resolution. For
those reasons, SEM is the imaging method of choice for many
investigators who work with materials having dimension. However,
because SEMs are used for the most part to reveal features
smaller than the wavelength of visible light, a microscopist who
uses an SEM to examine a larger structure—say, 8
millimeters wide, as in the large image at left—cannot
possibly get the entire sample into the field of view. Most of
the time, the researcher will take a series of images and
painstakingly stitch them together by hand in an application
such as Adobe PhotoShop. The process is tedious and
time-consuming.
Introducing John and Michael seemed obvious, and the results
were fruitful. I am convinced that such connections can advance
the way we visually document and represent research. I
welcome American Scientist readers to get in touch with me
if they have their own thoughts about connecting methodologies
which initially appear to come from different worlds. Below is
my own stitching attempt, this one of pieces of conversations
with John and Michael.
F. F. Michael, how does the program work?
M. C. The program finds common features in the
images and then aligns them by applying what is called a
"homography" to position and stretch each image before
blending them. Because the structures are fundamentally
three-dimensional, parallax makes it very hard to stitch them. To
get the best results John had to take lots of images
overlapping by, say, three-fourths of the image (ideally even more).
Imagine passing a video camera slowly over the sample. Then you can
rely on stitching together only the centers of the images, where
there will be less foreshortening.
F. F. John, if I remember correctly, some of
the initial attempts with other software would work only with images
taken from one vantage point, not one where you made images from
various planar views as you moved the SEM stage.
J. H. Yes, we first tried another application that
could only handle a fixed point. Then I tried another, with similar
disappointing results and besides, the code was much slower than
Acrylic. There was also lots of manual configuration—like
picking the common points to align the images—and under most
cases it wouldn't even stitch. Finally, as Michael suggested, I took
a number of overlapping images as I moved the SEM stage. Acrylic was
able to align the images automatically ... you just throw all the
frames on the canvas and hit "OK."
F. F. Why did you decide to image the sample with
an SEM and not an optical microscope?
J. H. First, the substrate is polished silicon and
looks like a mirror, so an optical image is badly confused by
reflections of the structures in the substrate. Second, the
structures are optically black, so an optical image wouldn't reveal
the curves and sharp edges of the structures, as seen in the
perspective view. Third, SEM is capable of much higher resolution
than an optical microscope by a factor of a few hundred.
F. F. Do you see any problems using this stitching program?
J. H. The stitching program worked very well with
these images. I was concerned that the program would have difficulty
aligning the edges of neighboring frames, because translating the
SEM stage to take each frame slightly changes the perspective. The
program slightly warped the frames to fit them together, but this is
hardly noticeable in the stitched images.
F. F. How do you feel about my cleaning the final
image for this article?
J. H. It looks nice; however, the bits you removed
are strands of nanotubes that grow from silicon chips left by
cracking the silicon wafer. These appear in the individual frames,
and each one consists of hundreds or thousands of nanotubes.