Thursday, May 10, 2012

More Evidence that Kids (American and Japanese) Prefer Accurate Sound Reproduction



Geoffrey Morrison, an audio writer at CNET and Sound & Vision has posted a nice summary  of my latest AES paper "Some New Evidence that Teenager and College Students May Prefer Accurate Sound Reproduction" presented at the recent  132nd AES Convention in Budapest, Hungary.


The paper is available for download here at the  AES E-library, and I have provided a YouTube video and a PDF of my presentation slides that summarize the main points of the research.


 The abstract of the paper reads as follows:


A group of 58  high school and college students with different expertise in sound evaluation participated in two separate controlled listening tests that measured their preference choices between music reproduced in (1) MP3 (128 kbp/s) and lossless CD-quality file formats, and (2) music reproduced through four different consumer loudspeakers. As a group, the students preferred the CD-quality reproduction in 70% of the trials and preferred music reproduced through the most accurate, neutral loudspeaker. Critical listening experience was a significant factor in the listeners’ performance and preferences. Together, these tests provide some new evidence that both teenagers and college students can discern and appreciate a better quality of reproduced sound when given the opportunity to directly compare it against lower quality options. 


The effects of culture and trained versus untrained listeners on loudspeaker preference are topics that have been discussed in previous postings on Audio Musings. To further shed some light on this topic, I also ran 149  native speaking Japanese college students through the same loudspeaker preference test along with 12 Harman trained listeners.  The graph below shows the mean loudspeaker preference ratings for these two groups of listeners along with the four different groups of high school and college students from Los Angeles.  




Not surprising, (at least to me) I found that the Japanese college students on average preferred the same accurate loudspeaker (A) as did the 58  Los Angeles students, and the trained Harman listening panel. The main differences among the different listening groups  were related to the effect of prior critical listening experience:  the more trained listeners simply rated the loudspeakers lower on the preference scale, and were more discriminating and consistent in their responses. This result is consistent with previous studies. The least preferred and least accurate loudspeaker (Loudspeaker D) generated the most variance in ratings among the different listening groups. This  was explained by its highly directional behavior combined with its inconsistent frequency response as you move from on-axis to off-axis seating positions. This meant that listeners sitting off-axis heard a much different (and apparently better quality) sound than those listeners  sitting on-axis.


 While the small sample size of listeners doesn't allow us to make generalizations to larger populations, nonetheless it is reassuring  to find that  both the American and Japanese students, regardless of their critical listening experience, recognized good sound when they heard it, and preferred it to the lower quality options.


It would appear that the reason kids don't own better sounding audio solutions has nothing to do with their supposed "deviant"  tastes in sound quality, but more do with  other factors  (e.g. price, convenience, portability, marketing, fashion) that have nothing to do with sound quality.  Music and audio companies should take notice that kids can indeed discriminate between good and bad sound, and prefer accurate sound, despite what the media has been falsely reporting for the last few years. With that out of the way, we should focus on figuring out how to sell sound quality to kids at affordable prices and form factors  they desire to own.


The research suggests that if we cannot figure out how to sell better sound to kids, we have no one to blame but ourselves. 

29 comments:

  1. The speaker test doesn't make much sense. Why would you put listeners off-axis with very directional speakers? People who buy a speaker like Martin Logan would listen to music mainly in sweetspot. Would the result been different if every listener listened on-axis?

    ReplyDelete
    Replies
    1. It makes perfect sense to test speakers both on and off-axis since that is how they will be used. If anything, we did Martin Logan a favor by including listeners off-axis since it scored worse for listeners sitting on-axis. The measurements indicate why: on-axis it's spectral balance favors the mids/highs making it sound harsh. The problem is ameliorated as you move off-axis.

      Delete
    2. It makes perfect sense to test speakers both on and off-axis since that is how they will be used. If anything, we did Martin Logan a favor by including listeners off-axis since it scored worse for listeners sitting on-axis. The measurements indicate why: on-axis it's spectral balance favors the mids/highs making it sound harsh. The problem is ameliorated as you move off-axis.

      Delete
  2. hi Anonymous,
    With over 200 high school/college students in this loudspeaker test we didn't have the luxury and time for testing them one at a time sitting in the same seat directly on-axis, like the Harman listeners did. Therefore, some of the students were on-axis, the others were slightly off-axis.

    Does this make the test invalid? Of course not since a) many/most? consumers don't sit directly on-axis and b) many of the reflected sounds arrive from off-axis angles. So a good loudspeaker should produce high quality sounds both on and off-axis regardless of its directivity so that it sounds good no matter regardless of where you sit or what the acoustics of the room are.

    In this case, the more directional loudspeaker (Loudspeaker D) was on average less preferred when listeners were sitting on-axis because it's spectral balance was too forward in the mid/treble region than it was for listeners sitting off-axis.

    ReplyDelete
  3. Hello,

    Couldn't it be said that rather than the best speakers, the students preferred the speakers that were closest to the ones that were used to create the recordings ? What if the recordings were mastered with the D speakers ?

    What is important is that young people still recognize when the music is at its best.

    However, in relation with the post on room correction, are we on the way to a standardization / norm for the listening environments in recording studios ?

    Regards.

    ReplyDelete
  4. Dear etcetcetc,

    You raise a good point about the interaction between the preferred loudspeakers for reproduction and the similarity of loudspeakers used for the recordings. Of course, there are no standards for loudspeakers used in the recording industry, and we have no way of knowing which loudspeakers were used for the recordings used in these tests. However, if you examine the performance of some of the most popular, highly regards recording monitors ( e.g. JBL, Genelec) you will find that they are all aiming for the same target response as what we aim for in our consumer loudspeakers: flat on-axis and smooth off-axis, with smooth directivity.

    Speaker D has none of those characteristics (except smooth directivity) and it is a dipole. To my knowledge, there exists few if any professional dipole speakers. You will simply not find them being widely used to monitor and master recordings.

    Sadly, we are nowhere being close to standardizing on the monitors, acoustics or calibration of the two in the recording industry. Until that happens, the quality of recordings will remain highly variable.

    ReplyDelete
  5. Hi,

    Could you explain, based on the measurements, how loudspeaker B scored higher than loudspeaker C as without seeing the results I would have placed these the other way round!! Loudspeaker C appears to have a smoother Sound Power and Directivity and does not look any worse On Axis. What am I missing, some low Q resonances perhaps?

    Many Thanks in advance.
    Adam

    ReplyDelete
  6. Hi Anonymous,

    Based on listener comments, I believe Speaker C was less preferred to Speaker B because it sounded very thin in the bass and forward throughout the mids/highs. The on-axis/listening window measurements show there is about a very broadband level mismatch above 600 Hz between the miss/highs and the bass/lower miss. So, despite its smoother sound power and directivity relative to Speaker B, the direct sound has poor octave-octave balance. Clearly, the manufacturer decided to maximize midband sensitivity so the speaker would be the loudest on the retail floor, where speakers are typically not level-matched -- but at the expense of octave-octave balance.

    Speaker B has it's own set of problems. It was apparently voiced by marketing to have a "boom and tizz" character created by the emphasis (bumps) centered at 80 hz and 10 kHz. However, the octave -to-octave balance is generally better than Speaker C.

    ReplyDelete
  7. Dear Dr. Olive:

    Great post. I’ve been looking into preferences for some time now and found your blog. You have done exactly one boatload, plus or minus a lifeboat, of research here.

    I've collected some data on subwoofer sales from various countries (from a large, well-known company). I used this when researching preferences for who is spending how much on subwoofers.

    I plotted "subwoofer sales (number of subwoofers) per capita" and "Dollars spent on subwoofers per capita" charts for various European and North American countries. The data did not contain Japan, so it was not included.

    Admittedly, this data needs to be taken with a grain, or even a pound, of salt. There was no attempt made at normalizing for average disposable income levels. There was no attempt made at normalizing for tariffs, different pricing strategies around different countries, or different product line offerings. This was only one sales year (2011), so overall trend may be skewed by local short-term economic conditions. As I said – it (as most studies) must be taken with a pound of salt.

    Even with all the caveats above, this might tell us yet more about preferences. I must share that I'm coming from an engineering perspective, not a sales or marketing perspective. That said, I’ve developed a keen interest in economics – especially over the past few years.

    If I set up an automotive test drive with the peoples of various countries, and I give them an economy car, a midsize sedan, and a luxury sports car to drive, I’d bet a majority would prefer the luxury sports car overall. However, I think it might be even more helpful in looking at if a certain part of the world is more willing to voluntarily part with limited monetary resources to buy such a car than another part of the world.

    The word “preference,” as far as I’m concerned, indicates I am willing to forego purchasing x (an incrementally nicer house or nicer car or nicer cell phone) so I may instead purchase y (an incrementally more accurate audio system).

    I summarize as follows: I love Ferraris just as much as the next guy – just not enough to actually buy one.

    Your studies are fantastic for concluding that audio system manufacturers and recording engineers should not target a specific “sound” for a particular part of the world, since nearly everyone seems to agree that accurate sound is best. Everyone appears to have “champagne tastes” when it comes to speakers. However, do they buy champagne, beer, Ferraris, or great speakers with their limited budgets?

    How do you think the role of economics relates to audio system preferences?

    ReplyDelete
    Replies
    1. It relates, not for preferences, but for actual listening. habits.

      Delete
    2. <>
      That question is not something I've studied but it's clearly an important factor in audio purchase behavior. I'm sure people with less money and less space in their homes tend to buy smaller, less powerful systems but I have no data to support it. But as you point out, it doesn't necessarily mean they don't aspire to higher quality more powerful sound reproduction systems.

      Delete
  8. Very interesting! Thanks for taking the time to do this. I have two concerns that I was hoping to get your feedback on: Would it not be better to have speakers with similar types of drivers instead of having one electrostat, one horn, etc. How can you account for whether the listener simply has an aversion or a preference for an electrostat? Second, what about the room? Different types of speakers interact with the room differently. My DeVore Super 8s love room treatments, but my Magnepan 3.6s sounded much better without them.

    Great read! Look forward to reading more of your material in the future.

    ReplyDelete
  9. Any chance we'll see studies of higher bitrate mp3s vs. WAV or FLAC in the future? 128 is getting less and less popular these days while 320 kbps is preferred by many online music stores.

    ReplyDelete
  10. Anonymous,
    At this point in time. I have no plans to test higher bit rates, as there are bigger fish to fry as they say. I agree with you that 128 kbps downloads are becoming rare. But as more and more content is streamed from the cloud to portable devices, cars and home devices, lower bit-rates than 128 kbps are quite common. Perhaps this will improve as higher bandwidth content and signals become common everywhere.
    Cheers
    Sean

    ReplyDelete
  11. Anonymou

    Regarding your above post on sJune 11, 2012 7:50 AM:

    I would love to see you data on subwoofer sales. There is no doubt that economics/demographics are factors that play into consumer's purchase decisions. So we agree.

    When I talk about "preference" I am strictly speaking about listeners' overall sound quality preferences measured in double-blind controlled tests. This measure is very specific and can't predict consumer behavior when deciding on what audio product they buy.

    If sound quality is an important factor in consumer's audio purchase decisions (as CEA research studies repeatedly tell us it is), then I believe our tests provide among the most accurate and reliable indicator of how good the product sounds.

    Sadly, audio companies generally do a terrible job of providing consumers meaningful information about the sound quality of their products, despite the fact that market research says it's an important criterion when consumers select audio systems.

    ReplyDelete
  12. I see that Americans and Japanese prefer flat speakers. Have you tested any Russians or Koreans though?

    ReplyDelete
  13. Hi Eisenhower
    I've not done any formal tests involving Korean sound quality preferences although we design and sell audio products that aim to be neutral and they seem to sell well there.

    I have included Russian audio dealers/distributers in our blind loudspeaker tests and they tend to prefer the same speakers as everyone else. However, their tastes in Vodka may be different than others :)

    ReplyDelete
  14. Thanks for the article,
    I think it's interesting that the you move towards quality over quantity when it comes to music speakers and other patio accessories . For me, it's all about the quality.

    ReplyDelete
  15. Hi, Mr Olive,

    Is the following an accurate description of the Infinity P362 curves shown in your presentation? (taken, else inferred, in part from the Toole book descriptions and prior HK papers):

    \\\ The dark top line is on axis, and the gray one right below it is +/-30deg horizontal plus +/10deg vertical. The third curve is average response estimated for first reflections in typical listening rooms. And finally the bottom curve is an average of 70 anechoic responses measured at 10deg intervals on horizontal and vertical orbits at 6’, each response weighted according to the proportional area of the sphere represented by each measurement.

    1/20-octave resolution. \\\

    thanks much,
    David Moran

    ReplyDelete
  16. Very interesting thread there, especially because transducers are one of the most essential parts of a playback system!

    Indeed what is expected from a speaker system is to translate exactly and conservatively i.e. with a flat frequency response, fast clean transients and with no phase issues etc.
    It is reassuring that during the blind tests, this was appreciated by a majority of the test populations (and also that trained ears actually scored better than untrained ears).

    Now if we could all lobby the media production companies and mastering houses to ensure they preserve the dynamics of their output, it would be fantastic!

    A

    ReplyDelete
    Replies
    1. Professional matering houses doesn't need to be warned about dynamics, they are really aware of them. They have to do what their clients asks for, them being production companies or artists. If this implies destroying the dynamics they will try to do it in the less destructive way. Read the mixing advise they give on any of their web sites to get an idea. ME love quality sound as much as any of us.

      Delete
  17. Hi AnonymousFebruary 22, 2013 at 6:03 PM

    I totally agree with you. How can we best convince the recording industry to restore the dynamics of music?

    ReplyDelete
    Replies
    1. Loudness normalization algorithms are the more accepted solution against loudness war... They are already aviable for some players, but they should be adopted by many of them as their default setting for this being more effective.

      Delete
    2. Perhaps, but there are apparently ways to trick the loudness meters into thinking something is less loud than it truly is.

      Delete
  18. David,

    Yes those are accurate descriptions of the spatially-averaged curve from top to bottom: on-axis, listening window, first reflections, total radiated sound power. The two bottom curves are the directivity indices of first reflections and sound power. All curves are derived from 70 measurements taken at 10 degree increments around horizontal and vertical obits at 2 m. distance.

    Before we developed our own measurement system, (Harman Audio Test System or HATS) we used MLSSA and the choice of raw resolution was 10 Hz or 2 Hz. The graphs you refer to look like 10 Hz to me. We then smooth the raw resolution with 1/20-octave smoothing.

    Since HATS we now do a log chirp that gives us a resolution of 48 log-spaced points per octave. We can do any octave or fractional octave smoothing of the raw data.



    ReplyDelete
  19. Anecdotally, I can say that as a 43 year old who listens to a lot of music all the time and has been doing so for most of his life I find that more and more often I notice artefacts in lower bitrate compressed reproductions, where I never used to when these first became popular. So I grew up with vinyl and cassette, then went through a CD phase - but with pretty crappy equipment, then MP3 and now I often listen to any of these in combination. When 128kbps first became practical it was amazing and sound quality was not an issue but now where options like FLAC are totally viable and moreover when even a $20 pair of headphones sound like something you would have paid $200 for 15 years ago, the quality of reproduction becomes really notceable. I noticed this especially just the other day when listening to a DAB+ radio broadcast (which are generally terrible anyway) and the audio quality was so bad the speakers were hissing. I switched to the FM transmission and it was just so much better I seriously wondered what else they were doing to the record with the DAB+ transmission apart from converting it to 64kbps AAC. I know the only thing they put on the FM signal is a limiter. That's not an argument in favour of analogue over digital, it is however arguing against too much lossy compression.

    ReplyDelete