IATH Best Practices Guide to Digital Panoramic Photography: QTVR and Related Technologies

 

6. Panorama Creation (Part 2): Stitching and Rendering

6.1. Introduction to the stitching and rendering process

The process of compiling source images into a single panoramic image is commonly referred to as “stitching.” This generally involves two steps: stitching the images together and rendering the stitched images into the appropriate output file format. This chapter will guide you through converting source images into a final product, a panorama that will be used by students, scholars, architects, historians, etc.

Good project management is the key to successful post-production work. There are many considerations that need to be balanced, including budget, timeline, and staff workload. The process involves specialized skills and can be time-consuming and expensive. If a technical support staff is available, it would be wise to take advantage of it and any other resources offered by the commissioner's institution or school. While the photography might take just a few hours, post-production can take weeks. The time required varies tremendously, since it is dependent on several factors:

  • How much computer memory and processing power are available?
  • What is the size, resolution and compression of the images?
  • How well were the camera and tripod calibrated during the photography shoot?
  • What are the skill sets of the technical staff?
  • How much material will be processed?

6.2. Preparing raw data

Stitching and rendering a panorama requires common photo-editing software packages as well as a suite of specialized software. Here are some of the more popular software applications, listed according to a normal post-production workflow.

Image Browsing

Image Editing

Panorama Stitching

  • PTgui, developed by Helmut Dersch
  • Enblend (New House Internet Services)
  • QuickTime VR Authoring Studio (Apple is no longer distributing this software, but it may be available on older machines [Classic OS 9.x Mac platform])
  • REALVIZ Stitcher (REALVIZ)
  • VR Worx (VR Toolbox)
  • Panoweaver (Easypano)
  • Photovista Panorama (iSeeMedia)

  • Autostitch (University of British Columbia)

The features and strengths of some of the more popular applications will be covered in the next section.

6.3. Software

Post-production begins with organizing the raw data. This includes browsing and evaluating the quality and usefulness of the images. If the source images are on film, this is when they should be digitized. Be sure to maintain back-up copies of data throughout the process.

6.3.1 Browsing and Evaluation

Normal photographic coverage of a cultural heritage site can produce hundreds (if not thousands) of images. It is impossible to overstate the importance of proper file management. It is a vitally important task and requires planning and regular maintenance. If not handled properly, information may be lost, tasks skipped or repeated, files misplaced, and the entire project can become tangled up in knots. There are many software applications that can help organize large collections of images by viewing, sorting, and assigning keywords to every image, thereby creating a database-like structure. These applications are very useful for standard photography, but, since the end-product of a digital panorama project is in fact a composite of many images, they are not practical in this case.

An easier way to manage data organization for panoramas is to build and maintain an intuitive directory structure that is independent of the software. In computer science and in taxonomy this is referred to as a tree structure. The root is the top-level directory, which contains the entire project. The branches contain more detailed level of specificity (for example, countries). These levels contain ever greater levels of specificity (such as cities and towns and then individual buildings). This continues to individual nodes (the leaves) that hold images of a particular location or day of shooting.

There are two major components: file names and file control. File names help in tracking data, versions, and revision histories. File control is concerned with creating, copying, moving, deleting, and so on. Most operating systems allow a great deal of flexibility with file names, allowing very long file names with non-alphanumeric characters such as "!" and "#" or allowing names without a suffix to indicate file format (such as ".txt" or ".jpg"). While it may seem a good idea at the time to call an image "take#4," someone looking at the file later may have no idea what the subject was, how to open it, or why this particular version is different from takes #1-3. These kind of names can also destroy data: if files are transfered between machines with an FTP application (such as Fetch or SecureFX), the software may balk at non-alphanumeric characters or even corrupt the file. Image manipulation applications may have trouble opening files without a suffix that identifies the file's format, even if the file is in fact the proper type.

To avoid these problems, observe good practice and name files in a consistent and informative form. Use a name of no more than twenty alphanumeric characters and a suffix of three characters that indicate the file's format. If more than one person will be handling the images, establish agreed-upon naming conventions that allow everyone to know what information is stored where and what formats are used. Try to avoid obscure or complex file names, such as "jk113004_124_2.gif." Not only is this kind of name hard to remember and difficult to use, but the content and version information in the name is unreasonably obscure.

A good naming convention for the directories is also important. The images will go through several stages and the naming system should reflect the status of the files within each directory. It is a good idea to have a directory for files that will not be used as well as a directory for files that need extra processing, such as groups of bracketed images that will be used for high-dynamic range (HDR) compositing. For example, here is one possible system of five main image directories:

  • Unprocessed: files straight from the camera
  • Processed: images that have been processed (rotated, etc.)
  • Unused: images that will not be included in the stitched panorama
  • WIP: “Works-In-Progress,” such as HDR composites
  • Final: work files for stitching, as well as the final outputs: QTVR movies, cylindrical or spherical and cubical renderings

A browsing application (such as Adobe's Bridge) allows for easy drag-and-drop organizational ability without having to toggle back and forth with the operating system’s file system.

Maintaining an organized structure such as this begins during the production phase (see section 4.6), and maintained throughout the entire course of the project. At the end of every shoot, the photographer should upload all of the day’s images onto his or her computer and fit it into the directory tree structure right then and there while the information needed is still fresh.

Run a quick quality assessment of images for focus; balanced exposure; tourist traffic (especially on the edges of the images); lack of major dust and sunspots; completeness of the panorama (ensuring that there are no missing shots). If there are bracketed exposures, this is the time to choose which shots can be used.

6.3.2 Digitization and scanning

Many photographers choose to shoot their source images with traditional analogue film. This entails the extra step of scanning the images into digital form in order to proceed. A scanner with automatic slide or filmstrip feeder is the ideal tool.1 For large format negatives, use a high-resolution flatbed scanner with an illuminated back. Be sure to properly adjust the scanners settings before conducting the scan. (For more in-depth instruction on film scanning, see additional sources such as "Creating Digital Resources for the Visual Arts: Standards and Good Practice” by Visual Arts Data Service.)

6.3.3 Back-up of data

Any seasoned computer user knows that Murphy's Law prevails, especially regarding computer hard drives and external FireWire or USB units. It is crucial to save multiple copies of all work on multiple drives that are used solely for back up and are dedicated to only one project at a time. RAID drives (redundant array of independent disks) are popular for their robustness and ease of use, but suffer two major disadvantages: they are very expensive and come as a single unit in one geographical location. Using a series of multiple identical drives is more cost-effective and the drives can be stored in several physical locations. This strategy is more time-consuming, but it is a good idea to have at least two identical back-up drives supplementing the main working drive (i.e., the computer’s internal hard drive).

Back-up should occur weekly at minimum, if not daily. This schedule can be automated by a number of web-based backup services or simply done manually. If the drives are part of a network at a university, many IT departments will schedule automatic nightly or weekly back-ups.

6.4. Color management on the computer

It is essential to have a properly calibrated display when editing digital photographs. The colorspace, or ICC profile, of the editing software should also be consistent with that of the camera. Some common color profiles are sRGB, AdobeRGB. Photoshop CS2 will automatically detect the color space of the camera based on the metadata associated saved in the image file. There are many books available for in-depth coverage of this topic.

6.5. HDR and multiple exposure compositing

Dynamic range refers to the range of light that is visible to the human eye or a single image/photograph. The human eye can see a much wider dynamic range than a modern digital camera. When looking at a stained glass window, for example, the eye can see not only the color and details of the glass but also the tracery and framing around it. A single digital photograph cannot capture such a wide luminance range; either the window will be visible and the wall and tracery will be black (cold), or the wall and tracery will be visible but the window will have lost all its color and show up white (hot). This problem can be solved, to a large extent, with multiple exposures, which can be used to create a composite image during post-production. When shooting interiors and exteriors, especially with high-contrast areas such as strong light and shadows, or bright windows and dark walls, it is a good idea to bracket shots up and down 2/3, 1, or more F-stops. In extreme situations, increase the F-stop bracket or take multiple sets of bracketed photos.

The additional exposures will provide data that can later be manipulated in image editing software with layer or alpha masks, so that all areas of a photograph will be visible and areas that were previously too hot or cold can be corrected. Some applications include features for automating computer-generated HDR images based on a series of varying exposures taken from the exact same position. This often produces wonderful results, though a bit of extra work is needed to makes the images useful for the stitching software. (A standard DSLR camera creates 8-bit images. The resulting HDR images are 16-bit or 32-bit, and will need to be translated back to 8-bit before the stitching software will be able to read the files.) Another problem to avoid is when the levels of an image do not match an adjacent image in the panorama sequence, resulting in awkward stitching and visible seams. Creating composites by hand is often the more efficient way to produce the correct results.2

6.6. Stitching the panorama

There are many software applications available for stitching panoramas, both open-source and proprietary. Not all programs perform the same tasks: some are capable of producing a wide range of output, including cubic and spherical renderings, while others create just cylindrical panoramas. Some include the option to create object files, both single-row and multi-row objects. Each program has its own unique interface and stitching method, though all follow a similar workflow that looks something like this:

  • Import source images into the stitching environment
  • Align the images relative to one another by either overlapping two or more images or setting common "anchor points" — physical features clearly visible in each image.
  • Render the output.

Stitching can be a very finicky and time-consuming process, and often requires tweaking various settings to place the images precisely in line. If the panorama is part of a larger set of panoramas that will be interconnected, add any hyperlink hotspots just before rendering.

Before importing images it is often necessary to pre-process the images in a photo editing application, such as Photoshop. It is almost always necessary to rotate the image 90° so that it is oriented properly. When working with images taken with a fish-eye lens, it is necessary to "defish" them, that is, to convert them to rectilinear or cylindrical projection.3 Despite an attempt by certain commercial companies to patent methods for processing fish-eye images for panoramas, the technology has been around for much longer. Even certain rectilinear images, such as those taken with a rectilinear 14mm or 15mm lens, have a small amount of barrel distortion that should be corrected. Two common issues with wide-angle lenses are vignetting, a reduction in brightness toward the periphery of an image, and soft lens focus due to chromatic or spherical aberration. These can all be corrected or at least mitigated with a number of applications, including Photoshop, DxO Optics, and LensDoc (Andromeda Software). Wherever possible, use batch scripts (see section 6.7, below).

If a scanning or circuit-style camera was used to create their source image, the process is slightly different. The main advantage is that the only stitching necessary is to join the two ends of the panorama together. A well-executed image will have a small bit of overlap on the ends so that blending can occur. The zenith and nadir shots may be added if creating a cubic or spherical panorama. If no nadir shot was taken, consdier using the nadir to display information such as metadata, geo-reference, contextual info, location, author, and copyright.

6.7. Batch processing, scripts

When creating several panoramas, a great deal of time can be saved by batch processing. Instead of manually creating and naming folders, sorting and moving files, grouping them together, editing configuration text files by hand, etc., let the computer do the work.

Two Perl scripts written by Ken Stuart can be downloaded and used for this task: PanoImageSorter.pl and PTMacPanoImageSorter.pl. Both process the images in a folder, sorting them into subfolders in sets of images composing a node, and then into sub-subfolders within each node for sets of bracketed shots. The latter adds the incorporation of a PTMac template file for each set of images, so that PTBatch can be used to create panoramas en masse. The scripts save a lot of time in manually creating a whole series of folders for a project, and then moving them all around appropriately, and then editing a template file to specify the location of each set of panorama images. The first script will run on any system with Perl installed, including all modern Macintosh computers. The second script uses a Perl module specific to the Macintosh and assumes the use of PTMac (a Macintosh-based image-stitching application).

In each script, a number of comment lines (starting with #) explain the variables expected (such as the path and folder containing the images, the number of nodes, their names, and the number of shots per node and the number of brackets per shot). To run a script, enter command-line mode, change folders to wherever you placed the script and type "perl PanoImageSorter.pl" or "perl PTMacPanoImageSorter.pl" after editing their variable values.

Before running PTMacPanoImageSorter.pl, the MacPerl module must be installed, even though Macintosh computers include Perl, so that the script may set the file type to be compatible with PTBatch. In order to install MacPerl, install the Apple Developer Tools software called XCode, which may require you to register as an Apple Developer but incurs no cost, and MacPerl from CPAN, a Perl module storehouse (the download link is in the upper right corner). After downloading and uncompressing MacPerl, open a Terminal session and change directories to the newly created folder. Once there, type "make test" and then, after it completes its tasks, type "make install" to finish the installation of MacPerl. Without MacPerl, the script can create copies of a PTMac template file, but PTBatch will not recognize the copies. Other utilities exist to change Macintosh file types, so if the installation and of MacPerl does not go well, place a # character at the beginning of the two script lines that reference MacPerl and use alternative means to change the PTMac template files' file type.

6.8. Rendering the panorama

A panorama can be rendered out as several different formats, called projections: cylindrical, equirectangular, also called spherical, cubical (six equirectangular cube faces), and any number of viewing formats, such as QTVR, Spi-V, and Shockwave.

Each stitching application has varying options for rendering, and each option has its own set of parameters. For example, when rendering to QTVR, set the window size (the size of the QuickTime player window), default pan, tilt, and zoom, and level of compression. For cylindrical, spherical, and cubical renderings the programmer can choose from many compression codecs, and can set the render area.

Some rendering engines are better than others, and there is external software that will improve the rendering process. For example, the open-source application Enblend may be a better choice than the blending software that comes integrated with the stitching application. However, Enblend can only be used in conjunction with certain projections (spherical and cylindrical) so it may be necessary to convert the output from one projection to another.

Sometimes the resulting image has flaws, such as tears, visible seams, ghosting, and artifacts. These can be corrected in image editing software. Ghosting, a common issue, occurs when a person toward the edge of an image moves before the next shot in the sequence can be taken. One way to fix this is to create a composite using one of the source images. Because straight lines look curved in a spherical projection, it may be easier to convert it to rectilinear cube faces, where the lines appear straight, before making the composite. In order to avoid the reduction of quality when a JPEG image is compressed more than once, it is good practice to render in uncompressed TIF format until the final rendering, which can be JPEG.

In the interest of long-term preservation, it is good practice to render out to several different formats, as well as storing the source images. A good preservation strategy will include source images and the final output (often QTVR), as well as spherical and cubical renderings. Preservation is discussed in depth in section 10 of this guide.

6.9. Hot spots, (hyper)links, and executable scripts

Most stitching programs offer the possibility of embedding “hot spots” within the QTVR movie. A hot spot is an area of the panorama that has been image-mapped, so when the mouse passes over the hot spot the cursor changes to indicate the existence of the hot spot. When clicked, any number of actions might be launched: loading a new panorama (navigational link), loading a detail image, loading a video file, opening a website, launching an executable script, etc. One of the most common uses of hot spots is in virtual tours that link a series of panoramas together. This is achieved by creating a single- or double-linked graph (the graph being the floor plan of a building, where the panoramas are the nodes, and the directional connections between the nodes are the hot spots). The advantage of a single-linked graph is that the user’s visit is directed. That is, one can proceed to the following panorama in a series of panoramas in a progression that is pre-programmed. This works well for presentations and editorial uses, such as a magazine web-feature. The usefulness of a double-linked graph is that the user is free to move about the space and move back and forth between nodes without having to restart the entire tour. This is ideal for individual study of a building.

There are many software applications available for developing virtual tours. Virtual tours are discussed further in section 8.6. Complex web development, however, is beyond the scope of this guide. The World Wide Web has a plethora of resources for web development. Two notable sites are the Apple Developer and Webmonkey. Sample HTML code, including <embed> tags, is available in Appendix 5.

6.10. Embedding author information, watermarks, sprite tracks, and copyright

Either as part of the rendering process, or after the QTVR has been created, it is possible to add annotations that are embedded into the file. This information can be viewed anytime through the playback application, such as QuickTime player or Cubic Navigator. Several applications, including QuickTime Pro and LiveStage Pro, allow the programmer to embed and edit annotations after the files have been rendered. Any number of the various annotations may be provided:

  • Name
  • Copyright
  • Description
  • Artist
  • Author
  • Creation Date
  • Software
  • URL Link
  • Information
  • Keywords
  • Comment

Adding redundant protections never hurts. It is good practice to embed a visible copyright on the panoramic image itself, as well as a watermark. A good place for a visible copyright mark (©) is at the bottom of the panorama, in the nadir shot, looking straight down. This is where the tripod stood, and the hole left in the shot is a great location for author and copyright information.

6.11. Converting historical images

Because the panoramic photographic process has been around for over 150 years, there are many examples of historical panoramas, found on many different photographic formats. The circuit camera creates one single panoramic image, while many photographers used standard cameras to create a series of images that comprise a panoramic view. Regardless of the format, any of these historical images can be digitized and converted into digital format. The tricky part is finding the right scanner for the job. A flatbed scanner can not scan images bigger than the scanner surface (8.5" x 11" for many models). For large jobs, a drum scanner, or handheld scanning device might be needed. Once the image is digitized, follow the steps provided in this guide appropriate for the specific format of the image, whether it is a series of overlapping images or one single panoramic image.

Notes:

1. The Nikon Super Cool Scan 4000 ED, for example. [back]

2. See Photoshop Masking & Compositing by Katrin Eismann, published by New Riders, for in-depth coverage on this specialized technique. [back]

3. You can download DeFish. [back]

Next section