Saturday, May 30, 2009

Harman's "How to Listen" - A New Computer-based Listener Training Program


Trained listeners with normal hearing are used at Harman International for all standard listening tests related to research and competitive benchmarking of consumer, professional and automotive audio products. This article explains why we use trained listeners, and describes a new computer-based software program developed for training and selecting Harman listeners.


Why Train Listeners?

There are many compelling reasons for training listeners. First, trained listeners produce more discriminating and reliable judgments of sound quality than untrained listeners [1]. This means that fewer listeners are needed to achieve the same statistical confidence, resulting in considerable cost savings. Second, trained listeners are taught to identify, classify and rate important sound quality attributes using precise, well-defined terms to explain their preferences for certain audio systems and products. Vague audiophile terms such as “chocolaty”, “silky” or “the bass lacks pace, rhythm or musicality” are NOT part of the trained listener's vocabulary since these descriptors are not easily interpreted by audio engineers who must use the feedback from the listening tests to improve the product. Third, the Harman training itself, so far, has produced no apparent bias when comparing the loudspeaker preferences of trained versus untrained listeners [1]. This allows us to safely extrapolate the preferences of trained listeners to those of the general untrained population of listeners (e.g. most consumers).



Harman's “How to Listen” Listener Training Program

Harman’s “How to Listen” is a new computer-based software application that helps Harman scientists efficiently train and select listeners used for psychoacoustic research and product evaluation. The self-administered program has 17 different training tasks that focus on four different attributes of sound quality: timbre (spectral effects), spatial attributes(localization and auditory imagery characteristics), dynamics, and nonlinear distortion artifacts. Each training task starts at a novice level, and gradually advances in difficulty based on the listeners’ performance. Constant feedback on the listener's responses is provided to improve their learning and performance. A presentation of the training software can be viewed in parts 1 and 2


Spectral Training Tasks

There are two different spectral training tasks. In the Band Identification training task, the listener compares a reference (Flat) and an equalized version of the music program (EQ), and must determine which frequency band is affected by the equalization (see slide 5 of part 2). The types of filters include peaks, dips, peak and dips, high/low shelving and low/high/bandpass filters. The task is aimed at teaching listeners to identify spectral distortions in precise, quantitative terms (filter type, frequency, Q and gain) that directly correspond to a frequency response measurement.


At the easiest skill level, there are only 2 frequency band choices, which are easily detected and classified. However, as the listener advances, the audio bandwidth is subdivided into multiple frequency bands making the audibility and identification of the affected frequency band more challenging.


The Spectral Plot training exercise takes this one step further. The listener compares different music selections equalized to simulate more complex frequency response shapes commonly found in measurements of loudspeakers in rooms and automotive spaces. The listener is given a choice of frequency curves which they must correctly match to the perceived spectral balances of the stimuli. This teaches listeners to graphically draw the perceived timbre of an audio component as a frequency response curve. Once trained, listeners become quite adept at drawing the perceived spectral balance of different loudspeakers, and these graphs closely correspond to their acoustical measurements [2], [3].


Sound Quality Attribute Tasks

The purpose of this task is to familiarize the listener with each of the four sound quality attributes (timbre, spatial, dynamics and nonlinear distortion) and their sub-attributes, and measure the listener's ability to reliably rate differences in the attribute's intensity. For example, in one task the listener must rank order the relative brightness/dullness of two or more stimuli based on the intensities of the brightness/dullness of the processed music selection. As the difficulty of the task increases, the listener must rate more stimuli that have incrementally smaller differences in intensity of the tested attribute. Listener performance is calculated using Spearman’s rank correlation coefficient which expresses the degree to which stimuli have been correctly rank ordered on the attribute scale.


Preference Training

In this task, the listener enters preference ratings for different music selections that have had one or more attributes (timbre, spatial, dynamics and nonlinear distortion) modified by incremental amounts.


By studying the interrelationship between the modification of these attributes and the preference ratings, Harman scientists can uncover how listeners weight different attributes when formulating their preferences. From this, the preference profile of a listener can be mapped based on the importance they place on certain sound quality attributes. The performance metric in the preference task is based on the F-statistic calculated from an ANOVA performed on the individual listeners’ data. The higher the F-statistic, the more discriminating and/or consistent the listeners’ ratings are --- a highly desirable trait in the selection of a listener.


Other Key Features

Harman’s “How to Listen” training software runs on both Windows and Mac OSX platforms, and includes a real-time DSP engine for manipulating the various sound quality attributes. Most common stereo and multichannel sound formats are supported. In “Practice Mode”, the user can easily add their own music selections.


All of the training results from the 100+ listeners located at Harman locations world-wide are stored on a centralized database server. A web-based front end will allow listeners to log in to monitor and compare their performances to those of other listeners currently in training. Of course, the identifies of the other listeners always remain confidential.


Conclusion

In summary, Harman’s “How to Listen” is a new computer-based, self-guided software program that teaches listeners how to identify, classify and rate the quality of recorded and reproduced sounds according to their timbral, spatial, dynamic and nonlinear distortion attributes. The training program gives constant performance feedback and analytics that allow the software to adapt to the ability of the listener. These performance metrics are used for selecting the most discriminating and reliable listeners used for research and subjective testing of Harman audio products.


References

[1] Sean. E Olive, "Differences in Performance and Preference of Trained Versus Untrained Listeners in Loudspeaker Tests: A Case Study," J. AES, Vol. 51, issue 9, pp. 806-825, September 2003. Download for free here, courtesy of Harman International.


[2] Sean E. Olive, “A Multiple Regression Model for Predicting Loudspeaker Preference Using Objective Measurements: Part I - Listening Test Results,” presented at the 116th AES Convention (May 2004).


[3] Floyd E. Toole, Sound Reproduction: The Acoustics and Psychoacoustics of Loudspeakers and Rooms, Focal press (July 2008). Available from Amazon here


52 comments:

  1. I hope you'll give some thought to publishing source code, so we could adapt it to Linux and other systems.
    --Greg

    ReplyDelete
  2. Greg,
    Actually, compiling a version of the software for Linux wouldn't require much effort.

    ReplyDelete
  3. Where can I get my OSX copy?

    Tim

    ReplyDelete
  4. Hi Tim,

    Thanks for your interest in Harman's "How to Listen." We are still discussing internally how and when to deploy it for the general public. When it's ready I will post a notice

    Cheers
    Sean

    ReplyDelete
  5. Thanks, Sean. I have a little arts blog, GrumpyOldArts. The audio section seems to be generating the most interest, so I'm thinking of focusing on it a bit more. I have no interest in measuring gear and publishing specs, etc., but a lively discussion of great recordings and the stuff we play it on, without the usual audiophile voodoo, would be of great interest. Your software sounds like something that would focus and facilitate that discussion. I look forward to its availability.

    Tim

    ReplyDelete
  6. I also would be very interested. Thanks for keeping us in the loop

    Randy B

    ReplyDelete
  7. Another one for having it available for the public.

    ReplyDelete
  8. Another one for having it available to the public.

    ReplyDelete
  9. Another one for having it available to the public.

    http://thewelltemperedcomputer.com/

    ReplyDelete
  10. That's a most impressive bit of software! I'd love to see it available too.

    Nicely done.

    -Caleb

    ReplyDelete
  11. This is exacly the kind of program I've been looking for. Ik sure hope you'll find a good way to make it available to the public. The already proposed web-based frontend would be a way of mutual benefit. I've used an ABX-comparator and files on the pcabx site, but its nothing like the program described above.
    Very much looking forward to it!

    Cheers,
    Martijn

    ReplyDelete
  12. I'd really like to get this software.

    +1 to make it available

    ReplyDelete
  13. I would realy love to see / hear this program in order to improve my hearing skills. If you decide to make this available to the general public, I will be very glad! Thanks in advance.

    Kind regards,

    Thom

    ReplyDelete
  14. Thanks for sharing this info. As an audio reviewer, I do hope Harman decides to make this tool public -- having a common descriptive language could really help communicate with audiences.

    ReplyDelete
  15. Hi Steve,
    Thanks for your comment. We hope to make the training software public before the end of this year. I agree that having a common descriptive language is not only a benefit to Harman listeners, but would be great tool for audio reviewers and consumers alike.

    ReplyDelete
  16. "before the end of the year"
    hip hip yes

    i am also interested in your aes paper for the new listening room
    i am aes member but cant find it in the Aes library

    ReplyDelete
  17. Hi Valentin,
    My AES paper on the Harman Reference Listening Room is currently only available on the 126th AES CD-ROM: http://www.aes.org/publications/conventions/
    AES does this so they sell more CD-ROMS, then they make it available as a single paper about 6 months after the convention. That means it should be available as a single paper by October.

    ReplyDelete
  18. Thanks i will be getting it then
    i am all sow interested in your 2nd part of equalizing loudspeakers in rooms co author with Martens
    o and the dsp correction paper is a must read for me

    thanks for sharing your time and wisdom

    ReplyDelete
  19. Sean

    I'm late posting; hope you're still reading...

    Having a common lexicon for listening would be a boon of unprecedented proportion. First, it would demystify the process. Second, it would eliminate the voodoo power-mongering at the so-called elite levels. Third, it would level the field for brands like Infinity, JBL, and Revel, who compete with art and science against the magical incantations and secret society logic of some brands and their priests and acolytes. Fourth, it would empower enthusiasts and audioplhiles (yes I use that word in its literal meaning) to make informed decisions based on sound principles. Fifth and finally, it might restore some fun and order to the whole audio pursuit.

    So, yeah, I'm encouraging Harman to share this with the masses. ;)

    ReplyDelete
  20. Hi DougNDO:
    Thanks for your comments. We are working on two things that we hope to make public: 1) The Harman "How to Listen" software that I talk about in this article and 2) an Audio Glossary that Floyd Toole is writing, and will include the subjective evaluation terms we use for listening tests.

    Cheers
    Sean

    ReplyDelete
  21. You know, I have this set of Dave Moulton's Golden Ears training CD's that I'm beginning to lend to SR staff to help them self-train their own ears to improve their ability to do their work. If there has ever been any usefulness or market for that CD set, then I must assume there will be even more so for your software. I realize that isn't the focus or purpose, but that's exactly why I'm pointing this out: it can serve more purposes than you might have thought.

    It can be especially handy for helping "technical volunteers" gain some critical hearing skills in such organizations as churches and community radio stations where these people are not otherwise "into" audio as much as the rest of us. Yet those are precisely the folks/organizations who would not have a lot of money to spend on such training. If you make this tool freely available, it can be a good thing for lots of folks. If you charge a license fee for it, consider a break for charitable orgs, please? :-)

    ReplyDelete
  22. Hi Zimbot:
    I am familiar with David's Golden Ear CD's and have a copy. He is giving a tutorial on in October at NY AES.

    Our plan still is to make the software available for free, and release it before December.

    Cheers
    Sean

    ReplyDelete
  23. Hi Sean,

    I must say, Harman is doing really great things lately. The LSR lines are great, as is the Vertec. Oh how far you've come in 10 years! I'm rather excited to try this software, and utilize these terms. Thanks for what you are doing!!

    -Travis

    ReplyDelete
  24. Will the software be available for windows as wll. Sounds very interesting..

    ReplyDelete
  25. Hi Sean,

    I just finished Floyd Toole's book and have enjoyed reading each of your posts. As a consumer who is both untrained in listening and measurement, I am very curious to know if the listening software you write about is available as a self-tutorial? I am also very curious to find out how one might accuately measure the in-room response of their speakers - I can see that my spectrum analyzer is really not the ticket - but what do you recommend for people to optimize sound in their own room? Is SFM an off-the-shelf product?

    I am a two-channel listener with hearing something like Dr. Toole has as outlined in his book. Sorry for the many questions...

    ReplyDelete
  26. Hi Nick,
    The "Harman How to Listen" listener training software is not yet available - but we hope to make it available soon. It is self-guided and doesn't require an instructor per se.

    I just posted a new article in thie blog about loudspeaker/room correction: see http://seanolive.blogspot.com/2009/11/subjective-and-objective-evaluation-of.html

    For room correction DIY's you need high resolution frequency measurements (1/20-octave resolution) with the ability to post-smooth the spatial averages. There are some low cost commercial measurement systems that can do this I think like ETF (PC) or Fuzz Measure (Mac). You need a calibrated 1/2-inch or 1/4-inch pressure microphone and stand that you can place at ear height on the listening seats(s). You need to do a spatial-average of the measurements since you don't want to equalize a peak or notch that is very location-specific. Then you need a 2-12 band parametric equalizer used to equalize the in-room response to a target function. If you read my recent blog posting and accompanying PDF presentation you will get a general idea of what the target curve should look like.

    ReplyDelete
  27. Looking forward for the software

    seams 2010 will be the release date

    ReplyDelete
  28. Just wanted to chime in with continued enthusiasm for the release of this software.

    I really have to commend Harmon et all for widely dispersing so much quality technical information. For a hobbyist like myself it's a great learning source.

    ReplyDelete
  29. I just found your blog, and I'm very impressed. Great reading!

    The software described here sounds excellent. Do you have an updated timeframe for when it might be released? And any news on whether there might be a binary (or even better, the source code) released for Linux?

    ReplyDelete
  30. Hi Smably,
    I think we are looking at around a March release for the training software. We haven't compiled or tested a Linux version but this could be easily done. At this point, there are no plans to release the source code.

    Cheers
    Sean

    ReplyDelete
  31. will you publish the specifics of the room treatments used in the most recent room?
    i am surrounded with revel ultima II product, and room conditioning is an ongoing process.
    Your insights are greatly appreciated.
    walt

    ReplyDelete
  32. Hi Anonymous,

    We are using 4 foot square (3-inch) thick fiberglass absorbers on the walls, some custom hemicylindrical diffusers and a few RPG Skylines to reduce flutter echoes. The rest is just furnishings and a removable 1.5 inch wool area rug with foam underlay. The diffusers were made using 2 ft diameter "Sono tube" cut in half with plywood on the back, lined with dynamat and filled with fibreglass.

    ReplyDelete
  33. Hello Dr. Olive,

    I too am much interested in your training software - any increase of objectviness in audio is much appreciated. I design audio equipment as a hobby, and testing is a big part of that - I live in LA too. Is there any chance to get a tour of Harman's testing facilities?

    Thanks much,

    Patrick Fleck

    ReplyDelete
  34. Dear Dr. Olive,
    As Indicated earlier I am realy looking forward to the release of this software. If you are looking for beta users I hereby volunteer. Is there any news on the (exact) date of release?

    Kind regards, Thom

    ReplyDelete
  35. Hi Thom,
    We are aiming for a public release in May.

    Cheers
    Sean

    ReplyDelete
  36. Count me as another reader excited for the release! I also would be happy to volunteer my time as a beta user/tester.

    ReplyDelete
  37. Would love to be a tester of the software whenever it comes out. I am a big fan of your work Sean and follow your blog. Please keep me updated with your progress on this.

    Thanks
    Sean M.

    ReplyDelete
  38. I'm still very much looking forward to the public release of the software! Too bad you weren't able to make it in May. Please keep us posted!

    Cheers,

    Martijn

    ReplyDelete
  39. Brew.Drink.Repeat.July 27, 2010 at 8:20 AM

    Really looking forward to this... any new updates on when it might be released?

    ReplyDelete
  40. I'm looking forward for the release of "How to listen" too! any updates?

    Regards,
    Oscar.

    ReplyDelete
  41. Hi,

    Do you know when "how to listen" will be available?

    Thanks

    JF Lejeune

    ReplyDelete
  42. Hello Mr. Olive,

    A while back I believe you said it is possible to get a tour of the Harman testing facility - how can we set that up? I'd love to come by.

    Thanks,
    Patrick Fleck

    ReplyDelete
  43. Patrick
    You can email me when you want to visit, and I can arrange it.
    Cheers
    Sean

    ReplyDelete
  44. Hello!

    Any updates on when the software will be released?

    ReplyDelete
  45. Scott:
    I've created a landing page for How to Listen here, and hope my software engineer has it today.

    http://harmanhowtolisten.blogspot.com

    ReplyDelete
  46. The page for How To Listen says that the software asks for four tracks, but only suggests three. Do we choose the fourth to our own fancy, was one omitted, or was the number required in error?

    ReplyDelete
  47. I can't seem to download the Mac version. Can you check the download link?

    ReplyDelete
  48. Kristian:

    The download link for the Mac version seems to work fine for me. Try this again:
    https://doc-10-18-docs.googleusercontent.com/docs/securesc/s696v3vk61e58p9j1o627vjuhvml4tgo/69gv122n6kddl0rnensa95d2rkvpk6p0/1327527900000/08175448946673264289/08175448946673264289/0B97zTRsdcJTfZTU3MGE3YjgtOWVmOS00ZTFjLThmNWItNjQ1YTZiMjY1ZTdj?e=download

    ReplyDelete
  49. Sean, Do you recommend wide dispersion speakers in normal asymmetric living rooms with a multitude of reflective objects of all shapes, sizes and distances and without any room treatment of any type? (Married)

    ReplyDelete
  50. Seth,
    Good question: In an asymmetrical room with wide dispersion loudspeakers you might get some lateral sound stage bias towards the stronger/earlier sidewall reflection, which could be ameliorated by absorbing the first side reflection.
    But generally, the first arrivals will dominate the localization of the auditory images with the reflections only adding spaciousness/image widening.

    ReplyDelete