Sunday, January 11, 2009

What Loudspeaker Specifications Are Relevant to Sound Quality?



This past week I attended the International Loudspeaker Manufacturer’s Association (ALMA) Winter Symposium in Las Vegas where the theme was “Sound Quality in Loudspeaker Design and Manufacturing.” Over the course of 3 days there were presentations, round table discussions, and workshops from the industry’s leading experts focused on improving the sound quality of the loudspeaker. Ironically, the important question of whether these improvements matter to consumers wasn’t raised until the final hours of the symposium in a panel discussion called: “What loudspeaker specifications are relevant to perception?”

The panelists included myself, Steve Temme (Listen Inc.), Dr. Earl Geddes (GedLee), Laurie Fincham (THX), Mike Klasco (Menlo Scientific), and Dr. Floyd Toole (former VP Acoustic Engineering at Harman), who served as the panel moderator. After about 30 minutes, a consensus was quickly reached on the following points:

  1. The perception of loudspeaker sound quality is dominated by linear distortions, which can be accurately quantified and predicted using a set of comprehensive anechoic frequency response measurements (see my previous posting here)
  2. Both trained and untrained listeners tend to prefer the most accurate loudspeakers when measured under controlled double-blind listening conditions (see this article here).
  3. The relationship between perception and measurement of nonlinear distortions is less well understood and needs further research. Popular specifications like Total Harmonic Distortion (THD) and Intermodulation Distortion (IM) do not accurately reflect the distortion’s audibility and effect on the perceived sound quality of the loudspeaker.
  4. Current industry loudspeaker specifications are woefully inadequate in characterizing the sound quality of the loudspeaker. The commonly quoted “20 Hz - 20 kHz , +- 3 dB” single-curve specification is a good example. Floyd Toole made the observation that there is more useful performance information on the side of a tire (see tire below) compared to what’s currently found on most loudspeaker spec sheets (see Floyd's new book "Sound Reproduction").

For the remaining hour, the discussion turned towards identifying the root cause of why loudspeaker performance specifications seem stuck in the Pleistocene Age, despite scientific advancements in loudspeaker psychoacoustics. Do consumers really care about loudspeaker sound quality? Or are they mostly satisfied with the status quo? Why do loudspeaker manufacturers continue to hide behind loudspeaker performance numbers that are mostly meaningless, and often misleading?

The evidence that consumers no longer care about sound quality is anecdotal, largely based on the recent down-market trend in consumer audio. Competition from digital cameras, flat panel video displays, MP3 players, computers, and GPS navigation devices, has decimated the consumers' audio budget. This doesn't prove consumers care less about loudspeaker sound quality, only that there is less available money to purchase it. Marketing research studies indicate that sound quality remains an important factor in consumers' audio purchase decisions. Given the opportunity to hear different loudspeakers under controlled unbiased listening conditions, consumers will tend to prefer the most accurate ones. Unfortunately, with the demise of the speciality audio dealer and the growth of internet-based sales, consumers rarely have the opportunity to audition different loudspeakers - even under the most biased and uncontrolled listening conditions. This is a perfect opportunity and reason for why the industry needs to provide new loudspeaker specifications that accurately portray the perceived sound quality of the loudspeaker.

So why is the loudspeaker industry not moving more quickly towards this goal? In my view, complacency and fear are the major obstacles. The loudspeaker industry is very conservative and largely self-regulated. There are no regulatory agencies to force improvement, or even check whether a product's quoted specifications are compliant with reality. Change will only occur as the result of competition, or pressure exerted by consumers, industry trade organizations (e.g.CEDIA, CEA) or consumer product testing organizations, like Consumer Reports. The fear of adopting a new specification stems from the realization that a company can no longer hide beneath the Emperor's new clothes (i.e. the current specifications). A perceptually relevant specification would clearly identify the good sounding loudspeakers from the truly mediocre ones. In the future, a perceptual-based specification like the one illustrated to the right, could provide ratings on overall sound quality, and various timbral, spatial and dynamic attributes. The consumer could then choose a loudspeaker based on these measured attributes.

In conclusion, all evidence suggests that consumers highly value sound quality when purchasing a loudspeaker, yet current loudspeaker specifications provide little guidance in this matter. It is time the loudspeaker industry grows up and realizes this. Adopting a more perceptually meaningful loudspeaker specification would permit consumers to make smarter loudspeaker choices based on how it sounds. This would better serve the interests of consumers and loudspeaker manufacturers who view the sound quality of a loudspeaker to be its most important selling feature.

32 comments:

  1. Great write-up. Thanks!
    Zarina
    Zarina Bhimani, ALMA International

    ReplyDelete
  2. Thanks Zarina. And thanks for breaking the ice as the first person to leave a comment -and a nice one at that :)

    ReplyDelete
  3. Harman's sitting on a goldmine here - they're one of the few loudspeaker companies on planet earth that cares how the speakers sound AND have the resources and will to build good-sounding speakers. Yet for some reason the average audio shopper -- experienced and novice -- has NO IDEA that there is such a significant a difference in how speakers are designed and built. A quick look at audio forums reveals that most experienced audiophiles consider the elements of speaker performance as similar to the elements in cooking: it's all a matter of taste. You might prefer spicy food, while I prefer less spicy. In the same way, many audiophiles talk proudly about how they prefer 'laid back' speakers or 'warm' sounding speakers or 'British' speakers. Surprisingly, these 'hifi' buffs claim to prefer speakers with colorations: i.e., distortions! This equation of speaker-auditioning with wine-tasting is one of the highest hurdles faced by 'science-based' companies such as Harman. As long as the typical consumer does not realize that there is an objective standard when it comes to sound, Harman's approach is not going to be seen as the "Gold Standard" that it is. Harman is going to be seen as 'one of many' with an interesting approach to R&D, instead of 'one of a handful'... or even "the only one" who has a good shot at hitting the target of accurately reproduced sound. It would be very helpful if the average opinion-leading audiophile agreed with points 1 & 2 in your post. I look at this as a wonderful marketing opportunity, and wish Harman would do an even better job of communicating how much the company is doing on behalf of music lovers. "Room friendly" was a good start, but more work should be done to communicate the sizable technical difference in Harman's approach and that of typical vendors.

    ReplyDelete
  4. If consumers can't trust spec sheets, and we can't trust Consumer Reports (yet), who can we trust? Wondering what you think of the speaker measurements done by various other publications, such as:
    *Tom Nousaine for Sound & Vision
    *John Atkinson for Stereophile
    *NRC for www.soundstage.com

    ReplyDelete
  5. And the second one will be positive to.

    I'm much looking forward reading this blogg.

    I hope this will make the science about sound, and sound reproduction a bit more easy available for us who won't dig in real deep in the JAES to often.

    I would be very glad to read more about the research done about how early reflections (walls, absorbers, diffusers etc) effect the perception of sound.

    Thanks.

    /gustav

    ReplyDelete
  6. Hi Mark,
    Thanks for your comments. I couldn't agree more with you, and will try my best to communicate the message that audio is a science --not to be compared with wine. There is no such thing as an "accurate chardonnay" whereas loudspeakers can and should be as accurate as possible in order to faithfully reproduce the recorded music (ie the art). I think of loudspeakers as being the wine glass, which wine experts say should be clear and transparent so that it doesn't misrepresent the true color flavor of the wine, which is the art.

    ReplyDelete
  7. Do you think a small select group of like-minded companies could band together and agree on a recommended set of metrics for mutual assessment and publish them?

    Do you think a standard could be recommended to industry reviewers that has consensus? What would it look like?

    In the testing performed by, say, soundandvision.com (NRC), do they have punlish the right data? If they do, how should a consumer assess it? What metrics should they use?

    Thanks for a great blog.

    ReplyDelete
  8. Thanks Eric for your compliment and questions. I can only speculate whether like-minded companies might band together and agree on some meaningful loudspeaker specifications. Today, even the companies that subscribe to comprehensive anechoic measurements with high frequency and spatial resolution don't generally publish these curves. The exception to this rule is JBL Professional (see the LRS http://www.jblpro.com/catalog/support/getfile.aspx?doctype=3&docid=568).


    There is some activity within the CEA to develop a more meaningful loudspeaker specification, but progress is very slow, and there are very few companies participating. Consumer Reports is moving in the right direction by doing comprehensive anechoic measurements and weighting the accuracy score on a combination of the direct and reflected sounds produced by the speaker. SoundStage.com uses the NRC to generate their loudspeaker measurements. I suspect most audio magazines will not or cannot afford to pay the $500-1000 to measure each loudspeaker at NRC or other independent testing labs like http://www.nwaalabs.com.

    The loudspeaker measurements in S&V are based on in-room measurements spatially-averaged over +- 30 to 60 degrees depending on whether it's a front left/right center or surround. The better speakers should have smooth and relatively flat curves. I've shown that in-room measurements with adequate frequency and spatial resolution do correlate with listener preferences- but not are as good as comprehensive anechoic measurements. A single curve cannot tell you the quality of the direct and reflected sounds.

    ReplyDelete
  9. Sean, thanks for the response and correction. I was thinking of SoundStage, and confusing it with S&V.

    It seems to me there are two problems, one is education of the consumer, the other is motivating better industry wide measurement methodologies and metrics.

    How about an advertising campaign, or even just a web site presenting a science driven data based shootout with your competitors? You have the data, why not leverage it? "We sound better, here is how we are better, here is exactly why".

    The point is you are in a unique position to address both requirments of consumer education and motivate an industry measurement methodology by, well, using a better one and publishing it. Power of the Internet and all that. Take the message direct to the consumer.
    Its a way to connect the science to a marketing return by changing how the consumer thinks about the problem. Maybe.

    Just a thought to consider.

    ReplyDelete
  10. Eric,
    I think the internet is an effective vehicle for educating consumers. A large percentage of audio consumers already use the internet to learn about loudspeakers before they make their purchase decisions. The first loudspeaker company or audio review magazine that puts up meaningful, trusthworthy loudspeaker specifications related to sound quality will be a magnet for consumers who want to purchase the best sounding products at a given price. Imagine the same effect that Wine Spectator or Robert Parker ratings have on the sale of a wine that receives high ratings -- except the loudspeaker ratings would be more reliable and objective.

    There are legal considerations that prevent putting a competitors' subjective and objective loudspeaker measurements on a web site against your own product> So the brands and model numbers of the competitors would have to be kept hidden. Ultimately the consumer will have to rely on unbiased organizations such as Consumer Reports to report who the best brands and models are. Time will tell whether this happens or not.

    ReplyDelete
  11. Good blog and a great idea. I really hope that you can garner support for what you are trying to do. A STANDARD for rational loudspeakers measurements would be great. But it must be available to all, or its not very useful.

    ReplyDelete
  12. So let's cut to the chase and get down to the $64,000 question:
    What is the flattest speaker you have tested?
    ;)

    Feel free to give one answer per target price range.

    ReplyDelete
  13. The JBL LSR 6300 series are among the flattest professional monitors I've tested. Check out the published set of anechoic measurements of the 6325P and the 6328P below. Speakers don't get much flatter and smoother than that. Add a subwoofer or 3, and some room-speaker calibration and you're all set.

    If recordings were made on speakers this good with proper room calibration we'd have fewer bad sounding recordings. If only the recording industry had standards - but that's a topic of it's own for another day.

    http://www.jblpro.com/catalog/support/getfile.aspx?doctype=3&docid=567

    http://www.jblpro.com/catalog/support/getfile.aspx?doctype=3&docid=568

    ReplyDelete
  14. Hi,
    I looked up for you on the web after reading "Sound reproduction", and here you are! Great blog! I would like to know more about those blind tests- method, reference, duration, type of listening material etc.

    Best regards,
    Vuki


    PS. looking forward to your next blog

    ReplyDelete
  15. Now all we have to do is apply this to automotive sound systems where we frankly do most of our listening, unless you are retired......

    ReplyDelete
  16. Hi Robert,
    It is true that according to CEA listening hours in automobiles almost matches listening hours at home (that was before gas hit $5 a gallon). As you know, car audio is a more challenging listening environment but I hope eventually there will be meaningful car audio specifications in terms of representing sound quality. This will depend on how well OEM audio suppliers can work together with the car manufacturers.

    ReplyDelete
  17. Hey Sean,

    Tom Nousaine loves those JBL 6300 series speakers and has them at his home. I bought some for surround use based on his measurements. Nice to see some agreement on those.

    Randy Bessinger

    ReplyDelete
  18. The JBL 6300s are undoubtedly fab, but a *leetle* pricey for a 5.1 setup.

    Behringer B203P monitors reportedly measure as pretty amazing and have very good build quality, for their price and size.

    testimonial about them from a speaker-building fan of Toole/Olive here:
    forums.audioholics.com/forums/showpost.php?p=480979&postcount=22

    ReplyDelete
  19. I know that I'm late to the party here, but this is a fascinating blog (I've RSS subscribed) and this particular article asks the question: What loudspeaker specifications are relevant to sound quality? I'd like an answer to that, please.

    ReplyDelete
  20. I think the answer is that none of the current loudspeaker specifications are very relevant -- but there is enough science to fix this.

    The relevant specifications would include measures of a) frequency response performance that characterize quality of direct,early, late reflected sounds b) a perceptual meaningful measure of nonlinear distortion and maximum SPL c) perhaps spatial quality (related to directivity). I think a) is already possible, c) needs some work, and b) needs even more work.

    ReplyDelete
  21. Sean,

    This is interesting and informative as usual, so I hope I don't take it too far afield. WRT published specs, I trust my ears more than my eyes, and I know what I like more than I know why I like it. Somewhat surprisingly, this led me to the JBL Synthesis® One Array system, despite my prejudice concerning horn-based systems.

    It sounded wonderful in the dealer's room. It sounded 100% better in my room after the DACS calibration. I was frankly amazed how much it improved beyond what I imagined.

    Any comments on DACS-like systems and what they do at the installation end of things to "improve" a speaker's performance? Does this in a way "change" a speaker's measurements?

    I know my questions are a bit obtuse, but I can't think of a better way to phrase them.

    Doug S

    ReplyDelete
  22. I think this is a very interesting debate, so let me join it with a slight delay ;-)

    You have already discussed the speaker with one of the flattest responses, the JBL 6325P. Unlike Steven Sullivan, I don't consider them too expensive, given that you can buy a unit for less than USD 500.

    My question is: Are there any speakers that "sound better" (in the sense of better scores on the subjective tests)? If not: Is there any reason to buy (or produce...) more expensive speakers, apart from the possibility of generating higher sound levels?

    ReplyDelete
  23. If "none of the current loudspeaker specifications are very relevant" - how can you proceed as a consumer to pick a loudspeaker that you like (assuming that you can't listen to all speakers, so you need some criteria to narrow down the list of potential candidates...)? I assume that according to current knowledge (a) is the only criterion that is "ready for use". But how reliable is this criterion if it is "not very relevant"? In other words: Do the loudspeakers with the flattest frequency response (like the JBL 6300 series) always score best in subjective listening tests, or are there occasions where the frequency response looks perfect, and yet the speaker does not sound "good"?
    For example, I have just looked at Toole's 1986 paper (part II, figure 8, page 332). To me (as a non-expert), the data for speaker 7 look very good, but this speaker only got a score of 7.4). Speaker has a similar flatness and similar deviations from a linear trend, and yet it scores much better at 7.9. Are there any explanations for this, or does it - in the end of the engineering process - always come down to endless listening sessions?

    Holger Hoffmann-Riem

    ReplyDelete
  24. Hi Doug S,

    The Synthesis DACS system does a multi-seat "room correction": it measures the response of each speaker at each seat, and then performs an equalization to reduce the variations in measured response across these 6 seats.
    We just submitted an AES paper on "The Subjective and Objective Evaluation of Room Correction Products". The short answer is room correction can significantly improve the sound of the unequalized speaker in the room --when done properly. Unfortunately, not all room correction products do things properly, as this paper reports. This will be a future topic on my blog.

    Cheers
    Sean

    ReplyDelete
  25. Hi Holger,

    Good question: "How do consumers pick a good loudspeaker given the trend of declining in-store demonstrations, and the misleading and incomplete specifications?" Most speakers that have excellent measured frequency response on and off-axis do well in our listeng tests (ie listeners give them high preference ratings). We can predict listeners' loudspeaker preferences with 86% accuracy based on anechoic measurements alone. So my advice is that you pressure loudspeaker manufacturers to publish or send you a full set of anechoic measurements to allow you to make a more informed, less risky purchase decision.

    The only exception to predicting loudspeaker sound quality based on measurements alone would be if the loudspeaker had high audible distortion at normal playback levels. As my article above discusses - traditional THD +N measurements are poor indicators of how the speaker sounds.

    I will have to look up the speakers you refer to in Floyd's papers to answer the 2nd part of your question. I will respond in a separate email.

    Cheers
    Sean

    ReplyDelete
  26. Hi Holger,
    Your question was: "Are there any speakers that "sound better" (in the sense of better scores on the subjective tests)? If not: Is there any reason to buy (or produce...) more expensive speakers, apart from the possibility of generating higher sound levels?"

    If the loudspeakers are competently designed, as the price increases you should more extended bass, and higher SPL output with less distortion. You may also get better controlled directivity since these designs employ multiple array drivers (versus an 8-inch 2-way for example). You should also get nicer looking industrial design, more cutting edge technology with things like active/amplified drivers, room correction,etc.

    As a general rule, there are diminishing returns in sound quality beyond a few thousand dollars -- particularly if the loudspeaker is designed by a small company that has limited access (or none) to sophisticated engineering and measurement tools. Unfortunately, the high-end is crowed with small companies like this which I think explains the increased variability in sound quality among different brands of "high-end" loudspeakers.

    Cheers
    Sean

    ReplyDelete
  27. Dear Sean

    Thanks for the two replies (and sorry for posting two similar comments - when the first one didn't appear after a day, I posted a second one...). I think you have now answered the most important questions, so there is no need for you to dig up the 1986 paper...

    In the meantime, I have done some research on the LSR 6300 series. The reviews are giving me the impression that the speakers sound as good as the frequency response looks. I am planning to set up a 5.1 system, so the 6300 might be the perfect speakers for me.

    Basically all I want to do is to listen to stereo music upmixed to 5.1. I assume that for a long time to come, most music will be recorded as stereo signals. Having read "Sound Reproduction" and having carried out a simple test with a passive matrix decoder, I am convinced that I will enjoy my CDs a lot more with a 5.1 system.

    However, there is one issue that is confusing me a bit. The 6300 monitors have digital inputs, so I assume that the best thing you can do with a digital signal is to feed it to the speakers as a digital signal. There are lots of surround processors on the market, but almost all of them focus on analog output - they usually have D/A converters and even act as preamplifiers. My impression is that I don't really need all those features when I use the 6300. Are there simple devices that just convert a digital stereo signal (spdif from CD drive) to a digital 5.1 signal so that I can set up connections to 5 monitors and a subwoofer? Since I am not in the recording business, I feel that I don't need any other features...

    It would be great if you could give me a hint!

    Best wishes
    Holger

    ReplyDelete
  28. Hi Holger,

    The JBL 6300 series only have analog inputs. It's the 4300 series that have AES/EBU SPDIF inputs. If you are listening at home to 2-channel material up-mixed to 5/7 channel I would use your receiver or surround processor for the up-mixing and feed the JBL LRS analog signals. Most surround processors don't have 5.1 or 7.1 digital outputs - just stereo digital outputs.

    Stereo music will be with us for some time, until the music industry can see how 7 channel music can make their wallets fatter. This is why you are still seeing development of new up-mixers. Unfortunately, the music industry run mostly by greedy, small minded, technological dinosaurs, who lack vision beyond projecting the growth of their personal bank accounts.If you have any doubts, read this book:http://www.amazon.com/gp/product/1416552154/ref=cm_li_v_cr_self?tag=linkedin-20

    ReplyDelete
  29. Hi Holger.
    The above link is not correct. The book I was referring to is: "Appetite for Self-Destruction: The Spectacular Crash of the Record Industry in the Digital Age" by Steve Knopper. It chronicles the bumbling and greed of the music industry over the past 35 years.

    http://www.amazon.com/Appetite-Self-Destruction-Spectacular-Industry-Digital/dp/B001NLKTA6/ref=sr_1_4?ie=UTF8&s=books&qid=1250965956&sr=8-4

    ReplyDelete
  30. JBLs are better or Infinity?
    :D

    Frankly, I am not trolling here - and I am sure many people would want to know this!

    ReplyDelete
  31. alpha1,
    I don't think you can generalize here, so it will vary depending on the model and price range. The Infinity Primus models are excellent value for the money. The more expensive JBL models offer more output.

    The design targets in terms of measured performance are generally the same: flat, extended frequency response on-axis with smooth response off-axis so you hear uncolored direct and reflected sounds in the room.

    Sorry I didn't give you the black and white answer you may have expected :)

    ReplyDelete
  32. Hi Sean, great blog!

    I noticed Revel speakers come with their own set of specifications like in-room response, target response, first reflection response and listening window response. The manual does add some explanations, but the in-room response still isn't clear to me. Can you clarify these specifications and are they relevant? Sadly, no graphs no the manual.

    Btw, the Revel F52 seems really good value if it really has +-0.5db response curves! These speakers might feel insulted when placed in my non-flat room. :)

    ReplyDelete