Binaural Room Scanning Part 2: Calibration, Testing, and Validation

In part 1 of this article, I described how binaural room scanning works and why it has great potential as a tool for psychoacoustic research and product testing. In part 2, I will describe some errors inherent to all BRS systems, which require proper calibration to remove them. Finally, I will summarize some research that has focused on testing and validating the performance of BRS systems .

BRS Errors

Unfortunately, all binaural record/reproduction systems inherently produce errors in the signals captured by the mannequin, and later reproduced through the headphones. The categories of BRS errors are summarized in Figure 2 [1]. Certain types of BRS errors (error 9) are easily removed with a correction filter. Individualized errors related to physical differences in the shapes and sizes of listeners’ ears/heads/torso versus those of the mannequin’s, are more challenging.

While it is possible to calibrate and remove individualized errors, doing so can be expensive and time-consuming, making BRS a less practical tool for psychoacoustic research and testing. Therefore, an important question is whether their correction leads to a significant perceptual improvement or difference in the listening test results. For example, if the error has no significant impact on the listening test results and conclusions, then the error is less of a concern. It is well known that humans can re-learn and adapt to errors in their vision or hearing introduced through injury or artificial means, suggesting that listeners may possibly do the same when listening through a BRS system.

BRS Calibration Testing and Validation

To answer some of the above questions, we have been conducting listening tests in parallel using both BRS and conventional in situ methods to determine whether they produce similar results. These tests have been conducted using different loudspeakers auditioned in a reflective listening room [1],[2], and having listeners evaluate the sound quality of different automotive audio systems [3]. So far, we have found no statistically significant differences in the results between the two methods. Listeners’ loudspeaker and automotive audio system preference ratings are the same whether measured in situ or through the BRS system. It is important to note that the BRS calibration used for these tests was based on a single listener, suggesting that individualized calibrations may not be necessary. Listeners are apparently adapting to and ignoring many of the residual errors that remain after calibration. We suspect adaptation is enhanced in multiple comparison listening tasks where the BRS errors are constant and common among the different loudspeakers or car audio systems being evaluated. Using a different BRS system, other researchers have reported similarly good agreement between BRS and in situ tests conducted on different audio CODECS [4], and an automobile audio system manipulated to produce different spectral and spatial attributes [5].

Future BRS Research

There remain many unanswered questions about the performance, calibration and testing of BRS systems. Is it necessary to capture and simulate the whole-body vibration that listeners feel when listening in a car or other listening space where the low frequency tactile element is significant? What is the best method for capturing and reproducing the non-linear distortion of the audio system, which is normally not included in the binaural room impulse response? Given that auditory perception is part of a multi-modal sensory experience, how important is it to include the visual cues (e.g. car and room interiors) that reinforce the auditory cues heard by the listener, and prevent cognitive dissonance? These are questions that we are currently investigating so that we can improve the overall accuracy and perceptual realism of BRS systems used in psychoacoustic research and product evaluations.

References

[1] Sean E. Olive, “Interaction Between Loudspeakers and Room Acoustics Influences Loudspeaker Preferences in Multichannel Audio Reproduction,” PhD Thesis, Schulich School of Music, McGill University, Montreal, Quebec, Canada, (February 2008).

[2] Olive, Sean,Welti Todd, and Martens, William L.,“Listener Loudspeaker Preference Ratings Obtained In Situ Match those Obtained Via a Binaural Room Scanning Measurement and Playback System,” presented at the 122nd Audio Eng. Soc., preprint 7034, (May 2007). Download here.

[3] Olive, Sean and Welti Todd, “Validation of a Binaural Car Scanning System for Subjective Evaluation of Automotive Audio Systems,” to be presented at the 36th International Audio Eng. Conference, Dearborn, Michigan, USA (June 2-4, 2009).

[4] S. Bech, M-A Gulbol, G. Martin, J. Ghani, and W. Ellermeir, “A listening test system for automotive audio - part 2: Initial verification [Preprint 6359]. Proceedings of the 118th International Convention of the Audio Eng. Soc., Barcelona, Spain, (May, 2005). Download here

[5] Søren Bech, Sylvain Choisel and Patrick Hegarty,”A Listening Test System for Automotive Audio – Part 3: Comparison of Attribute Ratings Made in a Vehicle with Those Made Using an Auralisation System,” [Preprint 7224], Proceedings of the 123rd International

Convention of the Audio Eng. Soc., Vienna, Austria, (October 2007). Download here.