New Methods for Quantifying Sonic Performance

Part One: Improved Methods for Estimating Sonic Differences in Audio Systems

Equipment report
Solid-state power amplifiers,
Tubed power amplifiers,
Solid-state preamplifiers,
Tubed preamplifiers,
Digital-to-analog converters,
New Methods for Quantifying Sonic Performance

In a previous four-part series of articles on computer audio we described a variety of factors that influence sound quality (abbreviated as SQ throughout this article) of digital music files (see TAS, December 2011 through March 2012; issues 218, 219, 220, and 221). We introduced a method by which a reader could replicate a reviewer’s audio testing experiences on his own system. Although this method was subjective, it was based on a series of very specific performance criteria (defined in part 1, issue 218) which could be quantified by a sonic measurement scale. This scale was defined as the difference in perceived sound between a CD and a high-resolution download or an SACD, both derived from a common master recording. In this way readers could replicate our experiments with their own equipment and subjectively hear what we heard. They could then judge for themselves the quantitative significance of our results as modified by their own system quality and perceptual skills.

In Part 1 of this new three-part series of articles, we introduce further refinements to our method of sonic evaluation. In the course of developing these improvements, we have identified what we believe to be an even better way to make sonic judgments. This new method can legitimately be considered objective, or at least quasi-objective, as compared to familiar subjective approaches. Among the benefits of this new method are its time-independence from one listening session to the next and a way of independently verifying subjective evaluation. For the first time anywhere, we present data that support this discovery. In Part 2 we provide three examples showing how these new methods can be used to determine the effect of three variables on system SQ: 1) effects of degree of FLAC compression on subsequent playback quality; 2) a surprising and inexpensive tweak which improves the sound of any high-quality audio system; and, 3) comparison of the effects of different equipment footers on the performance of a digital-to-analog converter. In Part 3, we use these methods to tackle the question of which is the better music server system, Mac or PC.

In our earlier articles on computer audio, we obtained some results which were a bit vexing and which we could not explain. In particular, we found that FLAC and WAV files did not sound the same. We also found that conversion of WAV to FLAC and back to WAV, particularly at CD-quality resolutions, also sounded different. In both cases, we found degradation of audio quality, despite the fact that all files we used were bit-identical. Some have suggested that such results could be explained by so-called “expectation bias.” We feel this possibility was excluded by strict adherence to the controlled, single-blind testing procedures we used throughout our listening tests. Some readers took issue with our methodology for quantifying the sonic differences and completely misunderstood the point of our measurement scale.

Despite theoretical criticisms leveled at our work and despite those who were unable to replicate our findings, many people did successfully reproduce and confirm our results. Now, some two-and-a-half years later, explanations for our observations are beginning to emerge involving conventional and non-conventional sources of jitter and additional, complex computer operational influences. We are gratified to see HDtracks change its policies and offer WAV downloads in addition to the normal practice of supplying compressed file formats. We hope that other vendors will follow this practice. There is also an indication that some download sites may be reducing FLAC compression ratios for improved sound. In addition to these changes in industry practice, we have been told that our results have had a marked positive effect on at least three software companies, and we have received compliments from many manufacturers around the world for the beneficial influence our article has had within the audio industry.

Description of Test Systems
System 1 – The signal path was computer server to M2Tech USB-to-SPDIF converter to PS Audio Perfect Wave DAC Mark II (PWD) to BSG qølTM processor to monoblock NuForce Ref 9SE V3 amplifiers to B&W 802 Diamond speakers. The processor also fed a pair of Velodyne ULD15-II subwoofers in parallel with the main speakers. All equipment was positioned on granite shelves supported by sand-filled steel stands. Cabling from the computer to the M2Tech was Wireworld Platinum. All other cables (interconnects, speaker, power and SPDIF) were from Exakte Audio Conductor which was chosen after comparison with a number of other brands because of its roughly 3 to 4 times greater performance-to-cost ratio over many other well-known brands. The SPDIF cable, which was 20 feet in length, was terminated with a custom-made adaptor in which the ground was isolated by a coupling transformer. AC power to the audio components was supplied from a 20-amp dedicated circuit which fed a PS Audio Power Plant Premier AC voltage regenerator/line filter. The computer was powered from an independent, dedicated 20-amp circuit. The PC used as a server consisted of a Gigabyte X58A-UD3R motherboard with 12GB of RAM, an Intel Quad Core i7 3.2GHz processor, and a 650W internal power supply in an Antec full-size tower case. It is important to note that this computer was dual purpose, acting both as a primary desktop computer and as a music server. As a result, additional hard drives, programs, and background functions were frequently running in addition to the software that supported the music server function. The operating system initially was Windows 7 Pro (64-bit) and was later upgraded to Windows 8 Pro (64-bit). The computer was supported by Nordost Bronze Sort Kones over a 2cm-thick granite base and powered with an Exakte Audio Conductor AC cable. Each of these external additions to the computer were tested individually and found to improve the sonic characteristics of the computer acting as music server. JRiver Media Center (JRMC) v. 18, and later v. 19, was used throughout the testing for music playback, with care taken to optimize all relevant output settings. Dimensions of listening room 1 measure ~32'x20'x7', with the speakers placed ~8' apart (center to center), ~5' out from the front of the room, the left speaker ~5' away from the left side wall, and the right speaker ~7' away from the right side wall.

System 2 – The signal path was dedicated computer server to asynchronous USB input on a PWD-II to BSG qølTM processor to mono-block NuForce Ref 9SE V3 amplifiers to Paradigm Signature S8 speakers. The processor also fed a pair of Velodyne SF15 subwoofers in parallel with the main speakers. The computer was connected to the PWD by a 1m long Wireworld Platinum USB cable. All other cables (interconnects, speaker and power) were from Exakte Audio Conductor. AC power to the audio components and computer was supplied from a 20-amp dedicated circuit which fed a PS Audio Perfect Wave Power Plant P3 AC voltage regenerator/line filter. The computer server used in this system consisted of an Asus P5K Deluxe motherboard with 6GB of RAM, an Intel Q6600 2.4GHz processor, and a 650W internal power supply in a Silverstone horizontal case. All of the equipment used in this system was supported on granite on steel shelves. Initially, Nordost Bronze Sort Kones were used under each piece of equipment. These were later replaced with Stillpoint Ultra Mini SS footers. The Windows 7 Pro (64-bit) operating system was initially used for listening tests and later upgraded to Windows 8 Pro (64-bit). A 500GB internal HD was dedicated for programs (later upgraded to a 240GB SSD when Windows 8 was installed) and a 1.5TB internal HD dedicated for music. This server was loaded only with the necessary software required to support the music server function. As in system 1, JRMC (v. 18 and later v. 19) was used as the primary playback software. Dimensions of listening room 2 measure ~23'x13'x7.5', with the speakers placed 6.5' apart (center to center), 4'10" out from the front of the room, and 3' from the side walls.

Results – Measurement Advances
Establishment and Description of a Better Measurement Scale for Estimating Sound Quality Differences

The sonic scale we created in our previous article (described in the Introduction section above) was based on a starting reference SQ of a CD played back from a PS Audio PWT transport compared with a high-resolution standard derived from the same master. We have revised the scale as illustrated in Figure 1 in view of the now more common practice of ripping CDs directly to a hard drive in WAV format. This has the advantage of rendering the scale more independent of the vagaries of different CD players or transports. This ripped file is now set to an arbitrary 100 points in SQ on this revised scale which allows for estimates of MP3 SQ falling below 100 points.

As described in Part 1 of our previous article on Computer Audio, a course measure of SQ was based on the number of A/B comparisons required to be certain of the differences in all the most important sonic qualities. To discern a small difference in SQ required 4 to 6 A/B comparisons; a medium difference required 3 to 4 comparisons; and a large difference required only 1 to 2 comparisons. In that article we chose a measurement scale that could be replicated by any reader on their own systems at home. The starting sonic marker of the scale was the SQ of a CD played back from a PS Audio Transport and DAC. The highest sonic marker was defined as the SQ of the authentic high-resolution filef rom which the CD was derived (available as a download).