Topics Related to Perception and Measurement of Reproduced Sound

On Tuesday, April 26th 2011, I will be giving a presentation at the meeting of the Los Angeles AES Chapter on several topics related to recent audio research at Harman International. The topics include:

• Preference in Quality of Reproduced Sound Among Generation Y
• Are There Cross-Cultural Preferences in Quality of Reproduced Sound?
• The Subjective and Objective Evaluation of Room Correction Products
• Harman's How to Listen: A Software program for Training Listeners

I've briefly discussed these topics in Audio Musings over the past few months, and you can find summaries of them by clicking on the links above. I'll be giving an update on new findings, and briefly touch on topics not mentioned above. As a door prize, Harman will donate a free copy of Dr. Floyd Toole's book Sound Reproduction (shown on the right side bar) autographed by the author of the book.

AES members and nonmember guests are welcome to attend. The meeting will be held at the Sportmen's Lodge in Studio City. More details can be found at the Los Angeles AES website.

The Subjective and Objective Evaluation of Room Correction Products

In a recent article, I discussed audio’s circle of confusion that exists within the audio industry due to the lack of performance standards in the loudspeakers and rooms through which recordings are monitored. As a result, the quality and consistency of recordings remain highly variable. A significant source of variation in the playback chain occurs from acoustical interactions between the loudspeaker and room, which can produce >18 dB variations in the in-room response below 300-500 Hz.

In recent years, audio manufacturers have begun to offer so-called “room correction” products that measure the in-room response of the loudspeakers at different seating locations, and then automatically equalize them to a target curve defined by the manufacturer. The sonic benefits of these room correction products are generally not well known since, to my knowledge, no one has yet published the results of a well-controlled, double-blind listening test on room correction products. To what degree do room correction products improve or possibly degrade the sound quality of the loudspeaker/room compared to the uncorrected version of the loudspeaker/room? Can the sound quality ratings of the different room correction products be explained by acoustical measurements performed at the listening location?

A Listening Experiment on Commercial Room Correction Products

To answer these questions, we conducted some double-blind listening tests on several commercial room correction products [1]. I recently presented the results of those tests at the 127th Audio Engineering Society Convention in New York. A copy of my AES Keynote presentation can be found here.

A total of three different commercial products were compared to two versions of a Harman prototype room correction that will find its way into future Harman consumer and professional audio products. The products included the Anthem Statement D1, the Audyssey Room Equalizer, the Lyngdorf DPA1, and two versions of the Harman prototype product (see slide 7). Included in the test was a hidden anchor: the same loudspeaker and subwoofer without room correction. In this way, we could directly compare how much each room correction improved or degraded the quality of sound reproduction.

Each room correction device was tested in the Harman International Reference Room using a high quality loudspeaker (B&W 802N) and subwoofer (JBL HB5000) (slides 8 and 9). A calibration was performed for each room correction over the six listening seats according to the manufacturer’s instructions. Two different calibrations were performed with the Harman prototype: one based on a multipoint six-seat average, while the second calibration used a six-microphone spatial average focused on the primary listening seat. The different room corrections were level matched for equal loudness at the listening seat.

The Listener's Task

A total of eight trained listeners with normal hearing participated in the tests. Using a multiple comparison method, the listener could switch at will between the six different room corrections, and rate them according to overall preference, spectral balance, as well as give comments (see slide 14). The administration of the test, including the design, switching, collection and storage of listener responses, was computer automated via Harman’s proprietary Listening Test Software. A total of nine trials were completed using three different programs repeated three times. The presentation order of the program and room corrections was randomized.

Results: Significant Preferences For Different Room Corrections

The mean preference ratings and 95% confidence intervals are shown above in Figure 1 (or slide 17). The room correction products are coded from R1 through R6 in descending order of preference. The identities of the products associated with the results are not relevant for the purpose of this article. Three of the five room corrections (RC1-RC3) were strongly preferred over no room correction (RC4). However, one of the room corrections (RC5) was equally rated to the no correction treatment (RC4), and one of the room corrections (RC6) was rated much worse. Overall, the sound quality of R6 was rated "very poor" based on the semantic definitions of the preference scale.

Perceived Spectral Balance of Room Corrections

Listeners rated the perceived spectral balance of each room correction across seven equal logarithmically spaced frequency bands. The mean spectral balance ratings averaged across all listeners and programs are shown in slide 18. The more preferred room corrections were perceived to have a flatter, smoother spectral balance with extended bass. The less preferred room correction products (R5 and R6) were perceived to have too little bass, which made them sound thin and bright.

Listener Comments on Room Corrections

Listeners also gave comments related to the spectral balance of the different room correction products. Slide 19 shows the number of times a particular comment was used to describe each room correction. The bottom row indicates the correlation between preference rating and the frequency of the comment. The most preferred room corrections were described as "neutral" and "full," which corresponded to flatter, smoother and more bass extended spectral balance ratings. The least preferred room corrections (R4-R6) were described as colored, harsh, thin, and muffled, which corresponded to less flat, less smooth, and less bass extended spectral balance ratings. Slide 20 graphically illustrates the same information in slide 19.

Correlation Between Subjective and Objective Measurements

In-room acoustical measurements were made at the six listening seats using a proprietary 12-channel audio measurement system developed by the Harman R&D Group. Slides 23 and 24 show the amplitude response of the different room corrections spatially averaged for the six seats (slide 23), and at the primary listening seat (slide 24). The measurements are plotted from top to bottom in descending order of preference, each vertically offset to more clearly delineate the differences. A few observations can be made:

1. The six-seat spatially averaged curves (slide 23) of the room corrections do not explain listeners' room correction preferences as well as the spatially averaged curves taken at the primary seat (slide 24). This makes perfect sense since all of the listening was done in the primary listening seat.
2. Looking at slide 24, the most preferred room corrections produced the smoothest, most extended amplitude responses measured at the primary listening seat. The largest measured differences among the different room corrections occur below 100 Hz and around 2 kHz where the loudspeaker had a significant hole in its sound power response. The room corrections that were able to fill in this sound power dip received higher preference and spectral balance ratings.
3. A flat in-room target response is clearly not the optimal target curve for room equalization. The preferred room corrections have a target response that has a smooth downward slope with increasing frequency. This tells us that listeners prefer a certain amount of natural room gain. Removing the rom gain, makes the reproduced music sound unnatural, and too thin, according to these listeners. This also makes perfect sense since the recording was likely mixed in room where the room gain was also not removed; therefore, to remove it from the consumers' listening room would destroy spectral balance of the music as intended by the artist.

Conclusions

There are significant differences in the subjective and objective performance of current commercial room correction products as illustrated in these listening test results. When done properly, room correction can lead to significant improvements in the overall quality of sound reproduction. However, not all room correction products are equal, and two of the tested products produced results that were no better, or much worse, than the unequalized loudspeaker. Room correction preferences are strongly correlated to their perceived spectral balance and related attributes (coloration, full/thin, bright/dull). The most preferred room corrections produced the smoothest, most extended in-room responses measured around the primary listening seat.

More tests are underway to better understand and, if necessary, optimize the performance of Harman's room correction algorithms for different acoustical aspects of the room and loudspeaker.

References

[1] Sean E. Olive, John Jackson, Allan Devantier, David Hunt, and Sean Hess, “The Subjective and Objective Evaluation of Room Correction Products,” presented at the 127th AES Convention, New York, preprint 7960 (October 2009).

Audio's Circle of Confusion

Audio’s “Circle of Confusion” is a term coined by Floyd Toole [1] that describes the confusion that exists within the audio recording and reproduction chain due to the lack of a standardized, calibrated monitoring environment. Today, the circle of confusion remains the single largest obstacle in advancing the quality of audio recording and reproduction.

The circle of confusion is graphically illustrated in Figure 1. Music recordings are made with (1) microphones that are selected, processed, and mixed by (2) listening through professional loudspeakers, which are designed by (3) listening to recordings, which are (1) made with microphones that are selected, processed, and mixed by (2) listening through professional monitors...... you get the idea. Both the creation of the art (the recording) and its reproduction (the loudspeakers and room) are trapped in an interdependent circular relationship where the quality of one is dependent on the quality of the other. Since the playback chain and room through which recordings are monitored are not standardized, the quality of recordings remains highly variable.

Creating Music Recordings Through An Uncalibrated Instrument

A random sampling of ones own music library will quickly confirm the variation in sound quality that exists among different music recordings. Apart from audible differences in dynamic range, spatial imagery, and noise and distortion, the spectral balance of recordings can vary dramatically in terms of their brightness and particularly, the quality and quantity of bass. The magnitude of these differences suggests that something other than variations in artistic judgment and good taste is at the root cause of this problem.

The most likely culprits are the loudspeakers and rooms through which the recording were made. While there are many excellent professional near-field monitors in the marketplace today, there are no industry guidelines or standards to ensure that they are used. The lack of meaningful, perceptually relevant loudspeaker specifications makes the excellent loudspeakers difficult to identify and separate from the truly mediocre ones. To make matters worse, some misguided recording engineers monitor and tweak their recordings through low-fidelity loudspeakers thinking that this represents what the average consumer will hear. Since loudspeakers can be mediocre in an infinite number of ways, this practice only guarantees that quality of the recording will be compromised when heard through good loudspeakers [1]. This is very counterproductive if we want to improve the quality and consistency of audio recording and reproduction.

Another significant source of variation in the recording process stems from acoustical interactions between the loudspeaker and the listening room [1]-[3] Below 300-500 Hz, the placement of the loudspeaker-listener can cause >18 dB variations in the in-room response due to room resonances and placing the loudspeaker in proximity to a room boundary.

Evidence of acoustical interactions has been well documented survey of 164 professional recording studios where the same high-quality, factory calibrated monitored was installed [4]. Figure 2 shows the distribution of in-room responses measured at the primary listening location where the recordings are monitored and mixed. The 1/3-octave smoothed curves show a reasonably tight ± 2.5 dB variation above 1 kHz. However, below 1 kHz, variation in the in-room response gets progressively much worse at lower frequencies. Below 100 Hz, the in-room bass response can vary as much 25 dB among the different control rooms! You needn’t look any further than here to understand why the quality and quantity of bass is so variable among the recordings in your music library.

Evaluating Loudspeakers When the Recording is a Nuisance Variable

Loudspeaker manufacturers are also trapped in the circle of confusion since music recordings are used by listening panels, audio reviewers, and consumers to ultimately judge the sound quality of the loudspeaker. The problem is that distortions in the recording cannot be easily separated from those produced by the loudspeaker. For example, a recording that is too bright can make a dull loudspeaker sound good, and an accurate loudspeaker sound too bright [5]. A review of the scientific literature on loudspeaker listening tests indicates that recordings are a serious nuisance variable that need to be carefully selected and controlled in the experimental design and analysis of test results.

At Harman International, we try to minimize loudspeaker-program interactions in our loudspeaker listening tests by using well-recorded programs that are equally sensitive to distortions found in loudspeakers. Listeners become intimately familiar with the sonic idiosyncrasies of the different programs through extensive listener training and participation in formal tests. In each trial of a loudspeaker test, the listener can switch between different loudspeakers using the same program, which allows them to better separate the distortions in the program (which are constant), from the distortions in the loudspeaker.

Through 25+ years of well-controlled loudspeaker listening tests, scientists have identified the important loudspeaker parameters related to good sound, which can be quantified in a set of acoustical measurements [6],[7] By applying some statistics to these measurements, listeners’ loudspeaker preferences can be predicted [8]. The bass performance of the loudspeaker alone accounts for 30% the listener’s overall preference rating. Good bass is essential to our enjoyment of music, which unfortunately is a frequency range where loudspeakers and rooms are most variable (see Figure 2). Controlling the behavior of loudspeakers and rooms at low frequencies is essential to achieving a more consistent quality of audio recording and reproduction. Fortunately, there are technology solutions today that provide effective control of acoustical interactions between the loudspeaker and rooms.

Breaking the Circle of Circle of Confusion

As Toole points out in [1], the key in breaking the circle of confusion lies in the hands of the professional audio industry where the art is created. A meaningful standard that defined the quality and calibration of the loudspeaker and room would improve the quality and consistency of recordings. The same standard could then be applied to the playback of the recording in the consumer’s home or automobile. Finally, consumers would be able to hear the music as the artist intended.

References

[1] Floyd E. Toole, Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms, Focal press (July 2008).

[2] Floyd Toole, “Loudspeakers and Rooms: A Scientific Review,” J. Audio Eng. Soc., Vol. 54, No. 6, (2006 June). A free copy of this paper can be downloaded here

[3] Sean E. Olive and William Martens “Interaction Between Loudspeakers and Room Acoustics Influences Loudspeaker Preferences in Multichannel Audio Reproduction,” presented at the 123rd Convention of the AES, preprint 7196 (October 2007).

[4] Aki V. Mäkivirta and Christophe Anet, “The Quality of Professional Surround Audio Reproduction, A Survey Study,”19th International AES Conference: Surround Sound - Techniques, Technology, and Perception (June 2001).

[3] Todd Welti and Allan Devantier, “Low-frequency Optimization Using Multiple Subwoofers,” Audio Eng. Soc., Vol. 54, No. 5, (May 2006). A free copy of this paper can be downloaded here

[4] Sean E. Olive, John Jackson, Allan Devantier, David Hunt, and Sean Hess, “The Subjective and Objective Evaluation of Room Correction Products,” presented at the 127th AES Convention, New York, preprint 7960 (October 2009).

[5] Sean E. Olive,”The Preservation of Timbre: Microphones, Loudspeakers, Sound Sources and Acoustical Spaces,”8th International AES Conference: The Sound of Audio (May 1990)

[6] Floyd E. Toole, “Loudspeaker Measurements and Their Relationship to Listener Preferences: Part 1,” J. Audio Eng. Soc., Vol. 34,No.4, pp.227-235, (April 1986). A free copy of this paper can be downloaded here

[7] Floyd E. Toole, “Loudspeaker Measurements and Their Relationship to Listener Preferences: Part 2,” J. Audio Eng. Soc., Vol. 34, No.5, pp. 323-348, (May 1986). A free copy of this paper can be downloaded here

[8] Sean E. Olive, “A Multiple Regression Model for Predicting Loudspeaker Preference Using Objective Measurements: Part II - Development of the Model,” presented at the 117th Convention of the AES, preprint 6190 (October 2004).