Friday, November 30, 2012

Behind Harman's Testing Lab

This past week I had an enjoyable time meeting well-known technology writer Robert Scoble who was visiting our Harman facilities in Northridge, CA along with his geek-in-command Sam Levine. As part of the tour, I showed  them our Reference Listening Room and Multichannel Listening Lab where we do product research and double-blind evaluations of loudspeakers. We discussed the science and philosophy behind how we design and measure the sound quality of our products.

One of the topics of discussion was my recent research that explores whether high school and college students from USA and Japan have different tastes and preferences in the quality of reproduced sound compared to older trained listeners.  We talked about differences in the tastes and performances of trained versus untrained listeners, and how Harman is able to accurately predict  subjective preference ratings of loudspeakers based on a predictive model that analyzes a set of comprehensive anechoic measurements.

After running Robert and Sam through a few trials of listener training using  our software "How to Listen", I decided to put them through a couple of double-blind listening test trials to see if they had the right stuff. They compared four different brands of floor-standing loudspeakers located  behind an acoustically transparent, visually opaque curtain where each loudspeaker is shuffled into the same position via an automated speaker shuffler. All of our tests are conducted double-blind because we have found that even trained listeners are influenced by nuisance variables such as brand, price, size, etc.

 In these tests Robert and Sam heard the same four loudspeakers that have been evaluated previously by hundreds of untrained listeners including young, old, American, Asian, and European listeners, whose preferences and performances were compared to those of our panel of trained listeners. From these tests, we have found evidence that most listeners prefer the most accurate, neutral loudspeaker regardless of age, culture or listening experience.

When the listening trials were done, the curtain went up, and Robert and Sam were surprised to discover their favorite choice was the most accurate loudspeaker which was the least expensive. The science works.  One of the speakers Robert didn't like was a model that he actually owned: it had excessive amounts of treble and upper bass, which I'm told is mandated by the manufacturer's marketing department who believe that "boom and tizz" are what their customers want. Luckily, I haven't met many of their customers, yet. Robert, then surprised me by turning on his camera doing an impromptu interview, which hopefully you'll enjoy. If you want to learn more about the engineering process and tools behind designing a speaker, check out the interview with one of our speaker engineering stars, Charles Sprinkle.

In my next blog posting I hope to discuss some of the exciting research we've been doing on the relationship between the perception and measurement of headphone sound quality. The goal is to develop the same science for measuring and predicting the sound quality of headphones that we've found useful for designing good sounding loudspeakers.  Stay tuned!


  1. Sean, that video is really, really cool. I am curious: How do listeners (and your testing) rank horn / waveguide speakers as compared to direct radiators? Is there a predictable preference there?

  2. Wow, just fascinating. How impressive to be able to ultimately predict end-user preferences based on speaker output. Cheers Dr. Olive!

  3. Hi Sean,

    have you seen this:
    Maybe an addition to your June, 2010 blog theme...

    Best regards,

  4. Chris PearsonFebruary 6, 2013 at 4:29 PM

    It's hard to generalize because there are well-designed and poorly designed horns and waveguides. What is important is that the directivity of the horn matches the woofers at the point they cross-over so you don't get a sudden hole or bump in the sound power or Directivity index.

    Listeners seem to like speakers with wide dispersion because the strong lateral reflections they produce increases sense of spaciousness and image widening. However, this only works if you have rooms with side wall reflections. It's less important for multichannel systems where the surrounds produce the lateral reflections for you (assuming the recording was well made).

    If you want to see an example of a truly radical horn/waveguide check out the new JBL M2 Reference Monitor. The measurements and sound quality are better than anything I've seen or heard in a Professional product.

  5. But aren't M2 measurements done with DSP included? Also, on-axis would be nice.

  6. Hi guys...

    Speaking of JBL measurements... JBL's line of home loudspeakers with horns and large waveguides (Studio and Studio LS series) looks really interesting. Somebody seems to be paying attention to directivity in home speakers :-)

    I can't seem to find any complete and objective measurements of those systems, though. Just the usual subjective reviews which I usually skip entirely.

    Is there a chance of obtaining some "real" data on those speakers?

    Kind regards,

    1. I have been encouraging our marketing people to publish meaningful measurements but it has been slow coming. If you want comprehensive measurements you'll need to purchase a JBL Pro speaker like a LSR. For example, the M2 has a complete set of anechoic measurements showing excellent response on and off-axis

  7. Hi,
    ponder over the prospect of future investment. It may be in fact worth saving up for an extra 6 months to get a more expensive model that will last longer. Also do your research on brands, which are the most affordable and provide the best quality and durability. Buying a medium priced system Likethat is more durable is a far better option than a cheap system that breaks down.

  8. Hi Sean,

    thanks for your work, it raises many questions and fights in my audiophile community, because it attacks on some long held beliefs :-)

    I'd be glad if you could briefly answer to some of following questions:
    - Is the quality of amplifiers important for comparing various loudspeakers?
    - What brand/type or amplifier class is being used in the testing lab?
    - Is the SNR ratio below audibility on all tested loudspeakers, or some more sensitive does emit hiss without music being played?

    I'm asking because many electrostatics loudspeakers present low-impedance capacitive load (<<2Ohm) to amplifier and most of amplifiers are not designed for such load. Also I've heard several (non-electrostatic) loudspeakers that did not play well with one amplifier, but much better with the other brand/type.


  9. "The least preferred and least accurate loudspeaker (Loudspeaker D) generated the most variance in ratings among the different listening groups. This was explained by its highly directional behavior combined with its inconsistent frequency response as you move from on-axis to off-axis seating positions. This meant that listeners sitting off-axis heard a much different (and apparently better quality) sound than those listeners sitting on-axis."

    In other words, you tested the speaker in a way that had nothing to do with the way a serious listener would actually set it up.

    Wide, even dispersion is not a meaningful speaker requirement (except possibly in AV systems). To weight it so heavily -- especially when one of the speakers is electrostatic -- is arguably malicious.