 |
| The predicted sound quality of 61 different models of in-ear headphones (blue curve) versus their retail price (green bars). |
During the interview, Kirk asked if it's possible to design a good sounding headphones for a reasonable cost. Or does one need to spend a considerable amount of cash to obtain good sound? Fortunately for consumers, my answer was that you can get decent sound without having to spend thousands or even hundreds of dollars. In fact, there is almost no correlation between price and sound quality based on our research.
I referred to the slide above that shows the predicted sound quality for 61 different models of in-ear headphones based on their measured frequency response. The correlation between price and sound quality is close to zero and, slightly negative: r = -.16 (i.e. spending more money gets you slightly worse sound on average).
So, if you think spending a lot of money on in-ear headphones guarantees you will get excellent sound, you may be sadly disappointed. One of the most expensive IE models ($3000) in the above graph, had a underwhelming predicted score of 20-25% depending what EQ setting you chose. The highest scoring headphone was a $100 model that we equalized to hit the Harman target response, which our research has shown to be preferred by the majority of listeners.
So, if you think spending a lot of money on in-ear headphones guarantees you will get excellent sound, you may be sadly disappointed. One of the most expensive IE models ($3000) in the above graph, had a underwhelming predicted score of 20-25% depending what EQ setting you chose. The highest scoring headphone was a $100 model that we equalized to hit the Harman target response, which our research has shown to be preferred by the majority of listeners.
The sound quality scores in the graph are predicted using a model based on a small sample of headphones that were evaluated by trained listeners in double-blind test. The accuracy of the model is better than 96% but limited to the small sample we tested. We just completed a large listening test study involving over 30 models and 75 listeners that will allow us to build more accurate and robust predictive models.
The ultimate goal of this research is to accurately predict the sound quality of headphones based on acoustic measurements without having to conduct expensive and time consuming listening tests. The current engineering approach to tuning headphones is clearly not optimal based on the above slide. Will headphone industry standards, headphone manufacturers and audio review magazines use similar predictive models to reveal to consumers how good the headphones sound? What do you think?
Once again you and your team provide solid evidence that the industry is driven by chaos when it comes to quality vs price. To pay 3000$ on IE headphones and not get the sound quality one would expect is just wrong and sad!
ReplyDeleteHope one day a universal standard will apply on headphones and speakers so we all can evaluate correctly the products and pay for what we choose instead of snake oil.
Until then, unfortunately, we will continue playing the Russian roulette when it comes to buy a product, or go for the few brands with strong research behind them and hope for the best!
Can't wait the results of your next research with on and over ear headphones posted on Facebook!
All the best from Romania!
Thanks Cristian. I think standardization of perceptually meaningful measurements of headphones and loudspeakers would benefit consumers as they would have clear indications of what sounds good and bad.
DeleteHi Sean Olive,
ReplyDeleteDo you think it's possible to equalize an IEM when in the ear whilst having a probe microphone also placed at the ear drum?
Would you also be interested in doing a Q&A on reddit /r/Headphones about your research?
You can contact the /r/Headphones moderators with a reddit account here:
https://www.reddit.com/message/compose?to=%2Fr%2Fheadphones
Other researchers have tried this approach and report some success. They argue that the variation in measured response amongst individuals is evidence that we need to equalize for every individual. However, I've not seen any evidence based on listening tests that such an approach would lead to signifiant improvement in sound quality of the individual. The majority of our listeners (n = 75) suggests that our target response is significantly preferred over the other 59 headphones they evaluated. The extent to which it could be improved from a measurement made at the individual listeners' ear drum remains to be seen (or heard).
ReplyDeleteAre you saying that equalization at the listeners ear drum to your target response curve from an ideal loudspeaker in a room would not see a significant improvement in listener preference versus equalization to the target response from a standard GRAS coupler?
ReplyDeleteAs every individual has unique pinna/ear canal geometry the frequency response of an ideal loudspeaker setup in a room at their own ear drum would not be the same as the frequency response heard at their own eardrum with a headphone equalized to your target curve on a GRAS coupler due to pinna/ear canal differences.
My concern is that your target frequency response curve is based upon an average head. I can understand why that average is important from a manufacturers perspective who cannot tune headphones to every unique head with its own pinna/ear canal. However, for me as an individual, the average head is not going to be close to mine in frequency response. The average only fits everyone the least poorly out of all possible frequency response targets.
I don't know if you've done research on this but have you evaluated listener preference of your ideal target curve measured at their own ear drum from a real loudspeaker setup versus the frequency response target equalized from a GRAS coupler.
We've equalized headphones and captured individual HRTF's of loudspeakers measured at the blocked entrance of the ear canal as others have (e.g. Smythe Realizer) -- but not at the ear drum for reasons I outlined above. So I don't know the answer to your question.
DeleteOur target started from a measurement of a loudspeaker in a room using coupler but we also did subjective experiments where listeners could adjust the bass and treble levels of the target over a broad range of frequencies. In the end, the average preferred levels were close to the original measurement
First, one has to show that the differences in each of us is significant or even exist. AFAIK, this is not supported by research but is largely person's own theory. If we had so drastically different senses, we would have trouble getting ANY averages. So all kind of "EQ:d to that particular ear" are theories.
DeleteI also feel that there is another factor, calling audiophiles average gets them defensive so they can't be, they have ot have better senses and when they encounter the first audiomyth that they can't explain, we get these theories, "well, my ears are different than your, thus i'm NEVER WRONG"... :) It is a topic that requires no responses, not until those differences are proven to exist at magnitudes that matter..
I agree with you. You can put probe mics in people's ears and measure differences when they are in diffuse sound field but how important is it perceptually when designing a headphone. We know the DF calibration is not preferred by most listeners but that is based on an average. The main issue with it is the flat bass response. Audiophiles may be the most opinionated listeners but they aren't always the most discriminating and reliable. Trained listeners are what matter for me
DeleteHello Sean,
ReplyDeleteJust discovered your and Floyd Tooles writings in my search for speakers for a modest or affordable home theater system. Struggling to find which speakers meet the criteria you both discovered? Can you point me to some models from Harmon? I have seen mentioned of he infinity primus 362 but it is no longer made. I have also seen mention of the JBL professional LSR 305 powered speaker. Which of the models including sub woofer from Harmon meet the criteria and are affordable? Struggling here..in sure what to buy. Thanks in advance.