Binaural Room Scanning Part 2: Calibration, Testing, and Validation

In part 1 of this article, I described how binaural room scanning works and why it has great potential as a tool for psychoacoustic research and product testing. In part 2, I will describe some errors inherent to all BRS systems, which require proper calibration to remove them. Finally, I will summarize some research that has focused on testing and validating the performance of BRS systems .

BRS Errors

Unfortunately, all binaural record/reproduction systems inherently produce errors in the signals captured by the mannequin, and later reproduced through the headphones. The categories of BRS errors are summarized in Figure 2 [1]. Certain types of BRS errors (error 9) are easily removed with a correction filter. Individualized errors related to physical differences in the shapes and sizes of listeners’ ears/heads/torso versus those of the mannequin’s, are more challenging.

While it is possible to calibrate and remove individualized errors, doing so can be expensive and time-consuming, making BRS a less practical tool for psychoacoustic research and testing. Therefore, an important question is whether their correction leads to a significant perceptual improvement or difference in the listening test results. For example, if the error has no significant impact on the listening test results and conclusions, then the error is less of a concern. It is well known that humans can re-learn and adapt to errors in their vision or hearing introduced through injury or artificial means, suggesting that listeners may possibly do the same when listening through a BRS system.

BRS Calibration Testing and Validation

To answer some of the above questions, we have been conducting listening tests in parallel using both BRS and conventional in situ methods to determine whether they produce similar results. These tests have been conducted using different loudspeakers auditioned in a reflective listening room [1],[2], and having listeners evaluate the sound quality of different automotive audio systems [3]. So far, we have found no statistically significant differences in the results between the two methods. Listeners’ loudspeaker and automotive audio system preference ratings are the same whether measured in situ or through the BRS system. It is important to note that the BRS calibration used for these tests was based on a single listener, suggesting that individualized calibrations may not be necessary. Listeners are apparently adapting to and ignoring many of the residual errors that remain after calibration. We suspect adaptation is enhanced in multiple comparison listening tasks where the BRS errors are constant and common among the different loudspeakers or car audio systems being evaluated. Using a different BRS system, other researchers have reported similarly good agreement between BRS and in situ tests conducted on different audio CODECS [4], and an automobile audio system manipulated to produce different spectral and spatial attributes [5].

Future BRS Research

There remain many unanswered questions about the performance, calibration and testing of BRS systems. Is it necessary to capture and simulate the whole-body vibration that listeners feel when listening in a car or other listening space where the low frequency tactile element is significant? What is the best method for capturing and reproducing the non-linear distortion of the audio system, which is normally not included in the binaural room impulse response? Given that auditory perception is part of a multi-modal sensory experience, how important is it to include the visual cues (e.g. car and room interiors) that reinforce the auditory cues heard by the listener, and prevent cognitive dissonance? These are questions that we are currently investigating so that we can improve the overall accuracy and perceptual realism of BRS systems used in psychoacoustic research and product evaluations.

References

[1] Sean E. Olive, “Interaction Between Loudspeakers and Room Acoustics Influences Loudspeaker Preferences in Multichannel Audio Reproduction,” PhD Thesis, Schulich School of Music, McGill University, Montreal, Quebec, Canada, (February 2008).

[2] Olive, Sean,Welti Todd, and Martens, William L.,“Listener Loudspeaker Preference Ratings Obtained In Situ Match those Obtained Via a Binaural Room Scanning Measurement and Playback System,” presented at the 122nd Audio Eng. Soc., preprint 7034, (May 2007). Download here.

[3] Olive, Sean and Welti Todd, “Validation of a Binaural Car Scanning System for Subjective Evaluation of Automotive Audio Systems,” to be presented at the 36th International Audio Eng. Conference, Dearborn, Michigan, USA (June 2-4, 2009).

[4] S. Bech, M-A Gulbol, G. Martin, J. Ghani, and W. Ellermeir, “A listening test system for automotive audio - part 2: Initial verification [Preprint 6359]. Proceedings of the 118th International Convention of the Audio Eng. Soc., Barcelona, Spain, (May, 2005). Download here

[5] Søren Bech, Sylvain Choisel and Patrick Hegarty,”A Listening Test System for Automotive Audio – Part 3: Comparison of Attribute Ratings Made in a Vehicle with Those Made Using an Auralisation System,” [Preprint 7224], Proceedings of the 123rd International

Convention of the Audio Eng. Soc., Vienna, Austria, (October 2007). Download here.

Binaural Room Scanning - A Powerful Tool For Audio Research & Testing

Binaural Room Scanning (BRS) is a powerful audio technology being used by Harman scientists to conduct innovative psychoacoustic research and listening tests that were previously not practical, or even possible. The roots of BRS are traced back to Studer (a Harman International company) who in the late 1990‘s developed a BRS processor that allowed recording engineers to remotely monitor their recordings via headphones through a virtual copy of their control room [1].

Unlike auralization methods, BRS provides an auditory display based on actual acoustical measurements of the loudspeakers and listening environment - not simulations based on a model of the loudspeakers and room. For this reason, BRS reproductions are significantly more accurate and realistic than model-based auralizations.

BRS measurements of the loudspeakers and listening space are made with an anthropomorphically accurate binaural mannequin equipped with microphones in each ear (see top photo above). Measurements are made at every 1-2 degrees over a range of ±60 degrees by precisely rotating the mannequin's head via a stepper motor controlled by the BRS measurement computer. Each measurement is stored as a set of binaural room impulse responses (BRIR) that provide the filters through which music is convolved and sent to a calibrated pair of high quality headphones (see bottom photo above). A key feature of the BRS playback system is its ultrasonic head-tracker: it constantly monitors the position of the listener's head, sending the angular coordinates to the playback engine, which in turn switches to the corresponding set of measured BRIRs. In this way, the BRS playback preserves the natural dynamic interaural cues, used by humans to localize sound in rooms. Without these dynamic cues, headphones tend to produce sound images localized inside or near the head with front-to-back reversals being quite common. Head-tracking is therefore necessary for accurate assessment of the true spatial qualities of the audio reproduction.

Current and Future Applications For BRS

As a research tool, BRS offers greater efficiencies and opportunities in how audio scientists research, develop and test audio products within home, professional and automotive listening spaces. BRS allows an unlimited number of acoustical variables to be manipulated, sequentially captured, and later evaluated in a highly repeatable and controlled manner. Using BRS, Harman researchers can do perceptual experiments and product evaluations that would otherwise be impractical or impossible using conventional in situ listening tests. This includes double-blind, controlled comparisons of different audio systems in different automobiles, concert halls or arenas, and home theaters.

BRS has already been used at Harman to study how the acoustical properties of the loudspeaker and listening room interact with each other, how these interactions affect the sound quality of the music reproduction, and the extent to which listeners’ adapt to the room acoustics when listening to multichannel audio systems [2],[3]. Over the next few years, BRS will help expand our current scientific understanding of how listeners perceive sound in rooms, so that we can optimize the sound quality of loudspeakers, acoustic spaces, and room-correction devices used to tame loudspeaker-room interactions. A BRS auditory display connected over the internet to a BRS database could even allow consumers to compare and select their most preferred loudspeaker model, concert hall seat, or automotive audio system configuration, without ever leaving the privacy of their home.

Finally, BRS brings enormous efficiencies, flexibility, and cost savings to psychoacoustic research and testing. The acoustical complexity of an automotive audio system can be captured and stored as a relatively small 10 MB file, which can then be emailed and evaluated anywhere in the world using a relatively inexpensive auditory display. The high costs associated with building expensive ITU-R listening rooms, transporting listeners, automobiles, and loudspeakers around the world for evaluation may soon be a thing of the past.

In the next installment, I will discuss some of the inherent errors found in all BRS systems, and how they can be removed through proper calibration. Some recent listening experiments will be described that validate the perceptual accuracy and performance of our BRS system.

References

[1] Horbach, Ulrich, Karamustafaoglu, Attila, Pellegrini, Renato, Mackensen, Philip, Theile, Günther, “Design and Applications of a Data-Based Auralization System for Surround Sound,” presented at the 106th Audio Eng. Soc. Convention, preprint 4976, (May 1999). Download here.

[2] Olive, Sean and Martens William L. “Interaction between Loudspeakers and Room Acoustics Influences Loudspeaker Preferences in Multichannel Audio,” presented at the 123rd Audio Eng. Soc., Convention, preprint 7196 (October 2007). Download here.

[3] Olive, Sean and Welti Todd, “Validation of a Binaural Car Scanning Measurement System for Subjective Evaluation of Automotive Audio Systems,” to be presented at the 36th International Audio Eng. Conference, Dearborn, Michigan, USA (June 2-4, 2009).