IATH Best Practices Guide to Digital Panoramic Photography: QTVR and Related Technologies

 

5. Additional Related Media

5.1. Object/inverse panoramas

Panoramas show a scene from a single anchored point in space looking out in all directions. Virtual reality (VR) objects can be thought of as an inverse of a panorama, where an object is at a single anchored point in the center and the viewer can examine the object from multiple points in space around it. Each successive frame depicts the subject from a slightly altered viewpoint, combining a series of images directed in at an object from many angles around it. This effect is achieved by either moving the camera around a stationary object or keeping the camera still and gradually rotating the object (e.g., placing it on a lazy Susan), capturing an image at each pause.

There is no minimum nor maximum number of images required to create a VR object, just as long as there is more than one image in the series. Of course, the more images used and the fewer differences between each frame, the smoother the rotation will appear. Rotation can be limited to show only a portion of the subject or it can be looped to show the subject in the round. Images can be captured around the subject within a single plane to form a single-row cylindrical object and then displayed in an application such as QuickTime Player.

Images from more than one vertical plane can be combined to create a multi-row object. These multiple rows can be arranged as a cylinder, keeping the line of sight parallel to the ground, simply raising and lowering the camera's height. They can also be arranged as a sphere or hemisphere where the camera swings up and over the subject, always pointing in at it, so that images in the highest row point straight down at the subject. Equipment is manufactured precisely for this purpose, where the subject is placed on a rotating surface, and a camera is mounted on an arm that swings in an arc up and over the object.1 This keeps the distance of the camera from the subject uniform in every direction, helping to keep the rotation smooth. Viewing this type of VR object in QuickTime Player is similar to viewing a single-row object, although in addition to moving the mouse left and right to change the rotation, moving the mouse forwards and backwards changes the vertical view.

Rather than using multiple object rows to change the vertical angle of view, you can choose to use them to change of state of the subject. An example is a clothing model photographed as a VR object by Will Rourke and Michael Tuite at UVa (Fig. 1).

Figure 1. This was part of a project for the University of Virginia Drama Department's collection of historic costumes. The initial default view allows the viewer to rotate the model around completely. As the user moves the mouse forward to advance the row, the top layer of clothes is removed. At this state, one can continue to rotate the object right or left to view the model from front to back without the top layer of clothes. With each advancing row, layers of clothes are removed until the model is shown in undergarments. By dragging the mouse backward, the layers of clothes appear again one at a time. No matter what the angle is around the model, the user can study varying layers of garments. Image by Michael Tuite and Will Rourke.

Small items, human models, sculptures and even entire buildings and structures can be depicted as VR objects. Architectural, site, and historical physical models also make good subjects. VR objects can be very useful for identifying, viewing, and documenting objects in context and in situ.

VR Toolbox’s VR Worx is a popular software tool for VR object creation, including single and multi-row objects. Because a VR object is a single file that contains information from many images, the file size can be quite large. While it is best to use full-resolution images to create the object, compress the image (via an image file format that compresses image data, such as JPEG) before exporting or rendering the object file, to minimize the file size, and to ensure that the window size is set properly.

When photographing the images that make up an object file, it is important to maintain proper exposure settings, especially when moving around a large object. It helps to capture multiple exposures, using bracketed exposures if desired, especially in areas where lighting conditions change rapidly. This will allow a smooth transition between exposure settings while moving around the object.

5.2. Audio

The addition of audio recorded on-site can greatly increase the immersive effect of a virtual tour. Environmental sound recorded both inside and outside of a site can help users grasp a space's acoustical properties, physical size, and setting. There are several ways to capture on-site audio.

Whenever possible, maintain high fidelity in audio recordings. The equipment should be an appropriate size for travel (such as a hand-held device), dependable, affordable, and readily available. It should also run on an independent power source, such as a rechargeable battery.

One popular choice is a small minidisk (MD) unit.2 The results are acceptable and come in stereo channels, the hardware is relatively inexpensive and runs on rechargeable batteries, and MDs hold a large amount of information and are available in many places. However, MD recorders use a compression scheme to reduce the size of data stored on the media, which reduces the resolution of the audio to a degree. For many purposes, though, MD audio is perfectly acceptable.

If you are already traveling with a video camera, it is a natural choice for capturing audio. MiniDV video cameras produce excellent audio recordings without employing the compression schemes that MDs use. Rechargeable batteries are standard and designed to last for long periods of time. If you trust a camera’s ability to capture video, then you can certainly depend on it when it comes time to capture audio. MiniDV tapes are inexpensive and can be found most everywhere. High-end miniDV cameras feature 2-channel XLR inputs for use with quality microphones. Note that using microphones that are not mounted on the camera's body reduces the risk of camera noise (the sound of the camera's motors and gears) appearing on the audio track. Capturing video and audio simultaneously can be useful even if the original plan was only to use the audio component. If the commissioner decides to use the video component as well, the audio context will be enhanced.

DAT (Digital Audio Tape) is perhaps the industry standard for high quality digital audio recording. Handheld, portable devices are available, and are very reliable. Often they will feature dual XLR inputs. The tapes hold an immense amount of data, and the results are top notch. The units can be less affordable, but, as the saying goes, you get what you pay for.

Another option might be to record audio with an iPod or a similar digital audio device. They are available with external microphones and can be famously large in capacity. It is worth investigating the emerging possibilities here.

It is also possible to record directly to a laptop computer, given the proper software and system capabilities (see section 5.4, below). There is a wide variety of possible hardware and software combinations for each platform. Check your system documentation for specifications and instructions.

The choice of microphone will change depending on the sound source. To capture the ambient sound of a room, use an omni-directional microphone. To hone in on a particular sound, such as the trickling of a fountain, use a directional microphone. When working outdoors, make sure to bring a wind shield fitted for the microphone. The presence of that little piece of foam can make the difference between good results and worthless noise.

Again, it is important to consider back-up equipment and plans. If replacement parts are needed, it helps to have a popular brand-name device, increasing the chances of finding the necessary bits abroad. Test the unit prior to departure to ensure it will deliver the desired results and run through the complete workflow, including audio capture, transfer of data from device to computer, exporting files, and post-production editing.

Capture sounds of interest that provide information about the acoustical properties of a space, such as an organ resonating in a cathedral. Pay attention to distinct contextual sounds that are unique to a site, such as bells tolling in an Italian piazza. Try to get an example of sounds that reveal the current use of the space. When embedded alongside a panorama, the sounds of the site bring the image to life.

5.3. Video panorama (moving image)

A video camera was one of the most popular methods for capturing digital panoramas when the technology was first introduced in the mid- to late-1990s. Set-up for this system is a matter of putting the video camera on a tripod and rotating the tripod head all the way around, making sure to pause at each click stop. The pauses must be long enough to avoid shutter blur; at least a momentary rest at each click stop. The camera’s automatic mode manages the image capture settings well enough, and there is little need for review after the shoot. After the camera has panned all the way around, the images can be imported by taking a screen capture at each pause, choosing the clearest frame available. Then stitch the images together into a panorama (see section 6.6).

Unfortunately, the downsides to using a video camera are numerous. Most important is the drastic reduction in image resolution when compared to a high-end digital SLR camera. Images captured from a video camera can produce cylindrical panoramas and spherical panoramas only if the video camera can be properly fitted to a spherical tripod head. Because a video camera is generally bigger, bulkier, and heavier that a DSLR camera, portability and convenience may be problems. In most cases, a video camera cannot fine-tune exposure settings as well as an SLR camera, and a bracketing function is not possible. This makes creating composite images in post-production more complex.

It is another thing altogether to include videos within a panorama. Those interested in enlivening static panoramas with areas of linear video can do so through authoring packages that offer this capability. Such software may or may not be available for a given platform and may or may not work with QuickTime, so options both for development and playback should be studied carefully. The video in this kind of development can be embedded within an authoring program (such as Flash or LiveStage Pro) on its own track and then activated through a clickable hotspot within the panorama.

Embedded video allows, for example, a television screen within a panorama to display video clips when a person clicks it in the panorama. The amount of time, skill, and effort needed to achieve a reasonably good effect of this nature will likely require serious commitment to mastering an authoring package. A less sophisticated approach would simply make the monitor clickable and launch a linear video in the full window or in another window.

In addition to flat linear video, developers using sophisticated authoring packages can embed animations, people, or objects shot on green-screen backgrounds. Combining any of these media types within a spherical panorama will require above-average skills with multi-media development packages. Search the Internet with phrases such as "embedding video within panoramas" or "video embedded in panoramas" to find the latest offerings of packages that offer these capabilities.

Currently the cutting edge of panorama development seems to be multi-camera linear video panoramas, videoramas, or motion VR. At present the results in this area usually end up at low-resolution and can be rather disorienting to the user. For an example, see An Inexpensive Panorama Video System, a cylindrical video panorama whose view can be changed at will. Other experiments simply point a single camera at a spherical mirror rather than stitching multiple video streams together. In any case, immersive interactive panoramas have not yet reached a quality level suitable for creating archival records or sustained or detailed viewing and may be limited to a cylindrical rather than spherical presentation.

In 2006, Clifford Ross from NYC, with assistance from Israeli brothers Liron and Tal Unreich from Tel Aviv, designed and tested the R2 camera, a high-resolution digital video camera that shoots 360° using nine cameras, nine mirrors, and nine microphones arrayed in a circle and mounted on a tripod. The R2 is a unique and experimental device, and its use is certainly not yet the norm in the industry, but it is another version of a video camera digital panorama system.3

5.4. Associated hardware and software

There are as many video cameras as there are budgets. Try to use the best one you can afford. Pay attention to the number of CCDs and their (combined) resolution, to the audio components, and to the quality of the results. If you only need to capture video for use on the web, there’s no need to buy an IMAX camera. Look for a camera with good manual controls and easy access to those controls. Avoid models that require navigating a menu system in order to get to the exposure settings, such as iris and shutter speed.

Using a video camera in conjunction with a steadying device or a Steadicam will increase the amount of useable video. Good results come from using a tripod, though in skilled hands a hand-held camera can also provide good video.

Video editing software is abundant. Choose the software that will do what you need it to do. Apple Finalcut Pro, Avid, Adobe Premiere, and iMovie are current standard software packages, each with various features and learning curves. If you intend to edit your own video, try to experiment with several applications first before investing your money and time.

Audio editing software is similarly diverse: ProTools, Digital Performer, Cubase, Apple Logic, SoundForge, Peak, GarageBand, iTunes, Audacity (freeware), and more. The same principals as above apply here. Look for software that meets your needs but does not place excessive demands on your time and budget. You are not preparing a recording of a full symphony orchestra: in most cases, you only need to crop the desired sound clips and export them as MP3 files. For this task, simple software will do the trick.


Notes:

1. Kaidan's web site has a page showing tips and possible set-ups for mounting the object and camera. [back]

2. The Minidisc Community Portal is an excellent resource about Minidiscs. Tape.com has a page of technical information and tips for recording media of all types, including MDs. Nickspicks.com is another useful resource. [back]

3. See also Nick Paumgarten's description of the R2 in action in "Bad-Ass Camera" in The New Yorker (21 Aug 2006). [back]


Next section