Apologies in advance if I don't respond to many other queries.



why have you not requested a retraction?

Short answer is that I do request corrections when anything I publish contains errors. But I wasn't the author of this, it was just a press release, 'important advantage' sounds to me like opinion rather than a factual claim, I can see how it could be used as a layman's phrasing of 'statistically significant', and scouring the internet would be impossible.





“I have eyet] to come across even one for hi res that does a decent job of doing level matched blind AB, leave alone a full protocol ABX… may I be pointed to even one ABX done in line with well established principle” – see the paper.

I have seen the paper. But is there any way to see a fully documented test itself? A link to one perhaps - that is the only way to it for most of us.

Unless I read the documented test, I can't conclude to how good it is. I may still not be able to do that, but it will be more than anything I have been able to find till now. By good, I mean how well single variable ABX protocols have been complied with.

Thank you in advance.



Most of the studies analysed in the paper were published by the Audio Engineering Society. They are all available from http://www.aes.org/e-lib/ , and can be downloaded freely by AES members.

One non-AES paper that can be downloaded by anyone is at https://www2.ia-engineers.org/iciae/index.php/icisip/icisip2013/paper/viewFile/160/146 . I don't by any means claim that this is the best, and it used an AB style paired comparison rather than ABX, but it should give you an idea of what a lot of the studies were like.

---

Thank you! The Japanese test seems to have definitive conclusions and I will read the methodology in detail to understand how it was done.

---

Apologies in advance if I don't respond to many other queries.

Short answer is that I do request corrections when anything I publish contains errors. But I wasn't the author of this, it was just a press release, 'important advantage' sounds to me like opinion rather than a factual claim, I can see how it could be used as a layman's phrasing of 'statistically significant', and scouring the internet would be impossible.





I'm sorry, but this is stretching the bounds of reality. No matter how many semantic arguments you wish to put forth, not even a layman would equate 'statistically significant' with 'important advantage'. I stand by my point; stating this study showed a 'important advantage' isn't justifiable marketing spin, or a translation to layman's terms; it is a flat out lie.



And If I were quoted as such when my results said nothing of the sort (in fact they say quite the opposite), I personally would request a retraction. I don't care it it was presented as an opinion, a fact, or an edict from God. I've seen no retraction, so I am left to think that you agree with the "layman's phrasing" which even to a layman, is a gross misrepresentation of the facts. This is pure marketing BS, and you seem to be a willing participant in that BS.



In short, it is astonishing that you would blame a mere misunderstanding, a marketing speak translation, or a dumbing down for laymen for what is in reality a lie, attributed directly to you. I'm left to wonder if AES's apparent support of all things Hi-Res has any bearing on your agreement with this "layman's phrasing".

---

One non-AES paper that can be downloaded by anyone is at https://www2.ia-engineers.org/iciae/index.php/icisip/icisip2013/paper/viewFile/160/146 . I don't by any means claim that this is the best, and it used an AB style paired comparison rather than ABX, but it should give you an idea of what a lot of the studies were like.

As you say, there are some issues with this report, and someone more experienced with ABX testing will do a better job of identifying these than I can. And what I point out may seem like nit picking to some, I believe it isn't for the following reason:



Achieving 90% of all that is needed to eliminate Bias does not mean there is only a 10% chance that Bias will colour the outcome - given how powerful and universal human biases are, that possibility can still remain as high as 100%. It really is necessary to come as close to a 100% elimination of bias to have a dependable result.



To start with the issues I see: first, the purpose of the study is such that tester bias cannot be ruled out.



Second, the test says it is a blind test, but I don't see that it is a double blind one. It isn't double blind AB, leave alone not being double blind ABX. Not being double blind, tester influence on the results cannot be ruled out.



I haven't been able to get enough clarity on the outcome that approx 57% prefer HR to CD, although it is clear that an outcome of no preference for either - as in both sounding the same - isn't possible to be obtained by the forced preference design of the test. Do 43% prefer CD to HR? I find that hard to swallow too! And do these numbers delivery any statistically valid outcome?



My understanding of human memory of sound quality is that it degrades in seconds and becomes unreliable. To overcome that, a rapid - less than a second - changeover between the tested samples is required to get a valid comparison outcome. This hasn't been employed.



Two important aspects DO seem to have been well addressed - mastering variability, and sound level matching.



So, while I have pointed out what my limited understanding tells me are flaws in test protocol, a robust peer review would do better in this effort, I admit, but would probably need access to the testers.



I don't plan to subscribe to AES, but if anyone can pick and describe any test there that is a robust ABX to determine the audible value of HR, it would be very interesting; till then, I will still maintain that I haven't seen any test that adequately rules out bias and takes into account human auditory issues including memory duration, and proves a preference for HR.



And if as you say a lot of the studies were like this, I am afraid that any meta analysis of these is just as flawed in its conclusions. To use a relevant analogy, no amount of subsequent effort and technology can overcome the defects in the source master.

---

It really is necessary to come as close to a 100% elimination of bias to have a dependable result... tester bias cannot be ruled out.

Good point and I fully agree.



I don't see that it is a double blind one.

Another hi-res evaluation paper by some of the same authors stated 'an operator was in a blind area outside the vehicle. It was not a double-blind experiment, but the subject and the operator could not communicate each other.'

I spoke with one of the authors for clarification on other aspects, and they used similar methodologies in both papers. So its likely only single blind, but with significant effort to reduce tester influence.



I haven't been able to get enough clarity on the outcome that approx 57% prefer HR to CD ah, that's the nature of these sorts of perceptual evaluation tests. They are actually well established throughout psychophysics (and applied to things like taste tests, perfume preference testing etc.), but its not your typical experimental and control groups as in evidence-based medicine. Think of the control as what would happen if participants were guessing. Then the outcome would be 50%. Note that a 50% result in a preference test with lots of trials does not necessarily imply that those participants can't discriminate, since it could be, for instance, that everyone can perceive the difference but half the participants prefer A and half preferred B. But a 100% result in those circumstances would be extremely unlikely to occur by chance.



For this experiment, standard error of the mean slightly overlapped 50%, so not considered significant. But it had a small number of participants and trials.



My understanding of human memory of sound quality is that it degrades in seconds and becomes unreliable - yes, and that would weaken any ability to discriminate. On the other hand, there is also the effect of auditory sensory memory. Studies suggest that it actually persists for quite some time. If the higher resolution content is played first, then one might retain the perception of that content if the lower resolution is played after (assuming that there's anything there to be perceived). And since we are looking at the limits of perception, long samples might be needed for listeners to identify differences. I grouped studies into those with short samples / quick changeovers vs long samples / long changeovers and the latter had much stronger results. But I wouldn't read too much into this since other differences between studies might have been the cause of this.





I haven't seen any test that adequately rules out bias and takes into account human auditory issues including memory duration, and proves a preference for HR. Even if people can hear a difference and even if that difference is due to higher quality, I doubt you'll find rigorous and general proof of preference. Such is the nature of subjective testing at the limits of perception and dealing with real world conditions. I'll give a couple of examples. In one study, participants were asked which audio format was best quality, and the live feed was the one most often ranked worst, suggesting that perhaps people didn't associate quality with purity. And a classic 1956 study suggested that those who have grown up listening to low quality reproductions may develop a preference for it.



if as you say a lot of the studies were like this, I am afraid that any meta analysis of these is just as flawed in its conclusions.

This is the big challenge with any meta-analysis, but luckily there is a lot of good advice from meta-analysis experts on how to deal with it. When I first started looking into this, it became clear that potential issues in the studies was a problem. One option would have been to just give up, but then I'd be adding no rigour to a discussion because I felt it wasn't rigourous enough. And its the same as not publishing because you don't get a significant result, only now on a meta scale. So I set some ground rules. I committed to publishing all results, regardless of outcome. I included all possible studies, even if I thought they were problematic (but discuss in detail potential biases). I decided that any choices regarding analysis or transformation of data would be made a priori, regardless of the result of that choice. And I did sensitivity analysis looking at alternative choices (like whether to exclude studies or analyse them differently).

One interesting thing that I found, which I did not at all expect, was that most of the potential biases would introduce false negatives. That is, most of the issues were things like not using high res stimuli, or having a filter in the playback chain that removed all high frequency content, or using test methodologies that made it difficult for people to answer correctly even when they heard differences.

---

And since we are looking at the limits of perception, long samples might be needed for listeners to identify differences. I grouped studies into those with short samples / quick changeovers vs long samples / long changeovers and the latter had much stronger results.



Such is the nature of subjective testing at the limits of perception and dealing with real world conditions.



I take your points; a couple of responses to the above quoted:

1. What about long samples/quick changeovers? You haven't addressed this method - would this not be better than either of the other two you mention?

2. The limits of perception reached by CD quality sound for it to be heard to be different/worse than HR and real world conditions that further compound this issue are not often considered by HR advocates. And far too often this difficulty is addressed by HR advocates by having a second variable in the mix, the HR master. Because that can often make the difference night and day, and this reason for that to be the case isn't easily picked up by the layman/market.

---

Hi everyone, and @joshr thanks for joining us! This is a great discussion, and I'd just like to remind everyone to respect the rules of the community and withhold from personal attacks or letting the heat of the discussion make you forget that this is a welcoming place where everyone is entitled to their opinion. Let's keep it friendly!



Thanks!

---

Hi Kumar,

1. Would be quite interesting. I think the optimal duration of timing and intervals would depend on which cognitive processes play the strongest role for a given task. But anyway, I don't think there was sufficient data for me to do a proper analysis on this. A lot of studies simply didn't give a lot of information on these durations.

And training seemed to be such a strong variable that it appeared to override everything else. Essentially (and I'm being loose with the language here), every study where participants were carefully trained in the task showed a strong ability to discriminate, every study where participants were untrained did not. And

---

... which made the effect of other factors like duration seem weak at best.



As for 2, you would clearly be comparing something very different if you look at commercial HD and CD quality recordings, if anything else was done in mastering besides just sample rate and bit depth conversion. But this argument works both ways of course. If the HD version is just an upsample of CD quality content, then its highly unlikely that there would be significant perceivable differences. Luckily, almost all studies avoided anything like this, and potential biases were noted and discussed when any such issues could have occurred.

---

And training seemed to be such a strong variable that it appeared to override everything else.



If training is guidance on a range of parameters to be observed/heard, with words suggested for the two ends of the scales for each, it seems to me that it would be useful; anything more starts running into contested territory!

But to my thinking, the forced preference is the bigger issue - and I would propose that it shows up in the CD V HR results where roughly there are an equal amount that prefer one over the other. More than anything else, I suggest that this shows there is little to choose between the two and people say - if I have to pick one, I pick this; and the fact that preferences expressed in that manner fall equally on either side of the divide has a message to convey about the nature of the outcome.

---

If the HD version is just an upsample of CD quality content, then its highly unlikely that there would be significant perceivable differences. Luckily, almost all studies avoided anything like this, and potential biases were noted and discussed when any such issues could have occurred.

I agree that in the study I was able to read about, this issue isn't a factor; but in the real world HR audio is almost aways sold after remastering the recording used for the original CD. Because - I suggest - this is the only way for it to sound audibly different in the real world of normal homes and listening environments, to make people that already have the original CD, to buy it again. And so that those that do not, buy it at a higher price than the original CD that is still on sale.



I doubt that there is anyone successfully selling HR music closely allied to the kind you refer to - where both the HR and the CD are being sold at different prices, but are recorded at the same levels from the same master, and just down sampled for the CD version.



IMO and to my mind this has been a good discussion that has reinforced for me the philosophy that real world improvement in sound quality can be obtained - if required for the enjoyment of music - only via paying attention to who is doing the mastering and to speaker quality and placement/room acoustics/room response DSP. That is where there are real improvements to be found.



Of course, when the day comes that the good masterings are only available in HR format, it will be a different matter. But as of now there are no trends that indicate that this sorry state of affairs will come to pass!

---

If the HD version is just an upsample of CD quality content, then its highly unlikely that there would be significant perceivable differences.



the kind you refer to - where both the HR and the CD are being sold at different prices, but are recorded at the same levels from the same master, and just down sampled for the CD version.





just to be precise, in my comment I referred to upsampling of CD quality content to a high res format. This was a potential criticism of the Meyer and Moran study, and could imply that some of the stimuli used in the test was not really high resolution.

Downsampled from high resolution to CD quality is a different scenario, and the studies primarily focused on just that sort of thing.

---

Unable to navigate to page 3 of this thread. InSided, lol. Maybe this post will fix...

---

Nope. But set thread to Newest First, which does show the final page. Screwy.

---

I simply have to say thank you to @joshr for joining in on this conversation! It is fantastic to have someone with your familiarity of this topic and depth of research connect with such a community!



As I'm sure you have noticed, many of the most active participants of this forum have built fairly strong views on this topic (and other topics :P) and as a result, may be perceived as being somewhat .... aggressive... in their response. 🙂 In my experience everyone on here does mean well and has a deep honest interest in the discussion, and you have added significantly to that conversation.



Incidentally, Having read much (with a lot yet to read) while I appreciate what people *may* be able to do in a testing environment, I stand by my original view that in a consumer market, higher resolution does not improve audio playback in any way that makes any meaningful difference to actual listening in a real world. I go back to the discussion referenced in the What Hi-Fi article : That people don't really note differences in their regular listening between mp3s and lossless such that they are not moving away from mp3s... I currently don't see the broader population demanding moving to higher-res as I don't perceive the typical individual truly discerning material differences in a real-world application. (And this is ignoring the potential communication bandwidth challenges that I perceive make product release/updates extremely problematic at this point in time.)

---

Yes, Hi Rez is much ado about nothing in the lives of the vast majority of us music lovers. Primarily of interest only to those trying to make a buck off of it, which includes a lot of shady companies and the magazines which support their claims without question. Objective journalism has no place in "high end" audio reporting these days, unfortunately.

---

And training seemed to be such a strong variable that it appeared to override everything else. Essentially (and I'm being loose with the language here), every study where participants were carefully trained in the task showed a strong ability to discriminate, every study where participants were untrained did not.



The issue of training is very interesting. Years ago a friend and colleague proudly showed me the stereo he bought as an undergraduate at Harvard in the late 1960s. Despite their perished surrounds and yellowing paper cones, he loved the sound. Having just moved overseas, I bought a new stereo, including some excellent Canadian Mirage speakers. I took them around to his place to compare the sound, and he still preferred his speakers! I left the speakers with him for a week, and when I picked them up he had new B&W speakers. It took a while for his ears to make the decision.



So the question is, to train or not to train and stay in ignorance? Which brings greater happiness? I don't have a clear answer, but I tend to prefer knowledge over ignorance. And I know my friend now enjoys orchestral music more because he can hear and discriminate all the instruments more clearly.

---

When someone says that to me that I am not able to pick out the differences between two bits of kit because I don't have the ears trained to do so, and my requiring some kind of test to rule out bias is unreasonable/unnecessary to those with trained ears, I draw my own conclusions about how easily swayed that person is.

---

I currently don't see the broader population demanding moving to higher-res as I don't perceive the typical individual truly discerning material differences in a real-world application.

Even if it could discern some differences, the broader population is far more interested in things like voice control, use of native apps, youtube video, home automation integration and the like. Even at the sacrifice of a little sound quality.



And who is to say it is wrong. For one thing, as the WHF interview says, stable streaming is more relevant to music enjoyment than marginal improvement in sound quality - music disruption can destroy the experience.



Secondly, the brain can effortlessly fill in for any little audible gaps that may exist - if they even do - between CD quality and Hi Res. But it cannot do so for things like voice control and all the other things that the broader population is now being offered. Sonos would be making a blunder if they did not take these trends and realities into account - and by all accounts, that is exactly what they are prioritising. They simply aren't interested in the market that cares less for these and more for hi res.

---

Good summary Kumar. Now let me ask you a question: when voice control becomes available, do you think you will use it? My concerns are twofold: one, I don't like the idea of a house filled with always-active microphones, and second, how does one add the next song to the queue over the sound of the previous song and without talking over the music?

---

I still think it is a gimmick. But I haven't used it, and ten years ago, I may have said the same thing about Sonos. In fact, I stayed away from wireless/internet audio for few years because I thought the loss of sound quality would be unacceptable to my discerning tastes in music!



So I will reserve a final judgement on voice until it matures fully and I have had a chance to test it. I believe that your question of talking over the music - as one example - has been addressed to quite a degree via the mic tech in use, and it will only get better in the right direction in future.

---

I will keep an open mind too, but I am seriously concerned I will no longer be able to play "what song is this" :)



By the way, I am currently listening to my only Hi-res DD5.1 recording on my Sonos 5.1 system. It's "Selling England By The Pound" by Genesis. It sounds great to be surrounded by sound (and the vocals stay up front while other instruments come from the rear, so genuine 5.1). The quality is excellent. I wouldn't know if the hi-res makes a difference or not, it doesn't matter. But what makes an enormous difference is Trueplay tuning the system. It really brings the sound to life!

---

Trueplay and even better versions of it are the future of sound quality improvements. But are you sure that it is Trueplay that is pulling of this trick - easy to determine by toggling it on/off. I suspect that the made for 5.1 master may be the one to assign most of the credit to.



And don't equate hi res with genuine mixed for surround sound audio. That isn't apples and oranges; surround sound mixes have a lot more audible content in the first place compared to 2 channel hi res. Somehow it never took off; perhaps because it can't be heard on the move?

---

I toggled it on and off. The sound went from vague to focused. It was a dramatic difference. I guess the mastering also makes a difference. The disc has stereo and 5.1 versions. I haven't compared them - I was just enjoying being surrounded by good sound. Interestingly, the disc doesn't have DD5.1, but has DTS 5.1. I'm having to play it on a Playstation to get it to work.

---

Based on what you found till now then, it would appear that Sonos is focusing on just the right places for improving your listening experiences!

---