Two years before Guadagnini’s death in 1786, King Louis XVI of France appointed a committee directed by Benjamin Franklin to investigate Franz Mesmer’s claims for the healing powers of “animal magnetism,” then being used to treat Marie Antoinette. “Mesmerists” were asked to identify trees, flasks of water, and other objects previously filled with “vital fluid.” They could not. This was the first blind-test on record, and it provides early evidence of what we now call the placebo effect.
Three decades later, scientists and musicians blind-tested a guitar-shaped violin by François Chanot against several by Stradivari and Guarneri. A panel of listeners judged the Chanot “not inferior.” Many more listening tests have since been done, with results often favoring new instruments. None, however, have much affected the near-universal belief in the superiority of Old Italian violins. But then few tests were conducted with sufficient rigor to yield scientific evidence, which by today’s standards requires double-blind conditions (where neither the subjects nor any researchers in contact with them know which violins are which) and publication in a peer-reviewed journal.
In 2007, French researcher Claudia Fritz, now at University Pierre and Marie Curie in Paris, started investigating how violinists evaluate violins. She was soon collaborating with Charalampos Saitis and Gary Scavone of McGill University. Convinced that blindfolding players disorients them enough to skew their perceptions, they decided to dim the lights and give the players darkened goggles. In a trial run at the 2010 VSA Oberlin Acoustics Workshop, the players were mainly violin-makers. I was among them, as was D’Addario string designer Fan-Chia Tao, with whom I co-direct the Workshop.
The experience impressed me. Without clues to an instrument’s identity, expectations about its playing qualities fall away. So too does a kind of confidence – or rather, false confidence – leaving an odd combination of uncertainty and liberation. Over the years I have spent many thousands of dollars on research equipment – none of it quite so effective as a pair of $6 goggles.
Old violins did not compare well with new ones at Oberlin, nor in subsequent experiments at McGill University – presumably due to the absence of first-rate old Italians. Tao and I were eager to work with Fritz on a larger experiment, but to make this credible, we needed distinguished instruments and high-level players. Fortunately, Glen Kwok, director of the International Violin Competition of Indianapolis (IVCI), visited Oberlin that summer, wanting to get violin-makers involved with the upcoming competition.
Claudia Fritz with the test violins in Indianapolis
So it was that Fritz, Tao, and I spent the best part of three days in a darkened, Indianapolis hotel room, listening to player after player try instrument after instrument. The results were published as “Player preferences among new and old violins” in the Proceedings of the National Academy of Sciences (January, 2012). Along with Fritz, Tao, and myself, the authors included Palmer Morrel-Samuels, who helped design the experiment, and Jacques Poitevineau, our statistician.
While the exact identities of the test instruments remain confidential, here are some very general descriptions: The three new violins, each by a different maker, were selected for their impressive playing qualities and contrasting tonal characters. Their ages ranged from several days to several years old. The old violins included a Guarneri ‘del Gesu’ (c. 1740), and a Stradivari (c. 1715) that has been used by a number of well-known soloists and chamber musicians. There was a second Stradivari (c. 1700) that was once the principal instrument of a well-known 20th Century violinist. This violin currently belongs to an institution that loans it to young artists; it came to us from a soloist who had used it for numerous concerts and several commercial recordings. Each of the old violins was in the long-term care of a highly qualified maker/restorer, and seemed in excellent condition. Use of the instruments came with the stipulation they remain in exactly the condition in which we received them. The combined value of the old violins was roughly ten million U.S. dollars, or about 100 times that of the new.
Most of the 21 players were involved with the IVCI, as contestants (four), jury members (two), or members of the Indianapolis Symphony (eight). Nineteen described themselves as professionals, ten had advanced degrees in music, and two were later chosen as competition laureates. Their own violins were (reportedly) between 3 and 328 years old, with values between eighteen hundred and ten million U.S. dollars. Players were scheduled for individual, one-hour sessions in a hotel room with relatively dry acoustics. (For initial testing, violinists often prefer dry rooms, where the violin sound is not too muddied by room reflections.) Modified welders’ goggles and reduced lighting made it impossible to identify instruments by eye. To mask any distinctive smells, we put a dab of scent under each chinrest.
A cloth screen divided the room into two areas. To preserve double-blind conditions, instruments were passed from behind the screen to a researcher wearing goggles, who laid the violins out on the bed. Players knew only that they would get to try a number of fine violins, including at least one by Stradivari. They were asked to bring their own bows. For the four who did not, one was provided.
In Part One of the experiment, players were instructed to choose their preferred violin from each of a series of randomized pairs. They had one minute to play whatever they liked on each violin of a pair, without switching back and forth between them. Unbeknownst to the players, each pair consisted of an old and a new violin. Our set of three and three thus yielded nine different pairs.
Players preferred an old violin an average of just 3.7 times out of 9. If this suggests a general preference for new violins, a closer look at the data shows that five violins were chosen about equally often, while a single violin was chosen less than half that often. Under these somewhat unnatural test conditions, only a conspicuously less-preferred violin seems to differentiate itself. That violin happened to be the c. 1700 Stradivari.
Part Two of the experiment was designed to emulate the way players choose instruments at a violin shop, where they typically try several before selecting one to take home on approval. All six test instruments were laid out on the bed.The players were given 20 minutes to play them in any manner and any order desired. They were then asked to choose (1) the single instrument they would “most like to take home with them,” and (2) the instruments they considered best and worst in each of four categories: range of tone colors, playability, response, and projection.
These terms were left undefined; if a player didn’t understand one clearly, he or she was told not to choose in that category. Though projection can by definition be judged only by listeners, players do routinely estimate projection, and it was their subjective impressions that interested us. Any number of instruments could “tie” for best or worst in a category. Players could also refrain from choosing at all. Eight players had difficulty deciding between two instruments; a violin that was almost chosen was recorded as “close-2nd.” Eight players spontaneously identified their least-favorite violins.
Figure 1: Number of times each violin was selected as take-home, then as best or worst in four categories. Above the take-home bar are the times each was chosen as close-2nd, and below the bar, as least-favorite.
Figure 1 shows the results. In contrast with Part One, a single new instrument (N2) stands out as most preferred. It was chosen eight times as take-home, three times as close-2nd, never as least-favorite, and three times as worst-in-a-category. The Guarneri was chosen five times as take-home, but also once as least-favorite, and nine times as worst-in-a-category. The c.1700 Stradivari remained the least popular – chosen once as take-home, once as close 2nd, six times as least-favorite, and 16 times as worst-in-a-category.
Each instrument was the take-home choice for at least one player. All except N2 and N3 were also the least-favorite for at least one player. This wide divergence in individual tastes carries through into the four categories: With the sole exception of N2’s projection and the playability of N3 & O2,each instrument was chosen as best and worst at least once in each category. Unsurprisingly, each subject rated their take-home violin as best in at least one category. New violins were rated significantly higher than old for playability and response, but showed no significant difference for tone-colors or projection.
Blind-tests with wines have shown a tendency for inexperienced tasters to prefer less expensive wines, while experts prefer more expensive. We saw no evidence of anything similar among our players. The take-home violins of both jury members were new (N2 & N3), and each juror made comments comparing their take-home instrument favorably with their own Stradivari or Guarneri ‘del Gesu’ violins. All in all, just 8 of 21 subjects (38%) chose an old violin to take home. While sample sizes are too small to draw general conclusions, this stands as a striking counter-example to prevailing beliefs.
At session’s end, the players were asked if they could guess the “making-school” of their take-home violins – an informal way of assessing their ability to distinguish new from old. Of the seventeen who responded, seven said they had no idea. Another seven guessed wrongly, saying an Old Italian was new, or vice-versa. Just three guessed correctly. One player’s comment underscores how new instruments can be stereotyped: “It definitely sounds like a modern violin because it’s very bright, doesn’t have a depth, doesn’t have a round ring.” It was a Stradivari.
These results raise many questions. How would preferences change with different players, violins, or test conditions? Do preferences formed in twenty minutes carry into the long term? More research is needed. Our results do help explain the failure of science to “unravel the mystery” of Old Italian sound. “Good” and “bad” playing qualities seem widely distributed among instruments, old and new. Moreover, what’s “good” for one player may be “bad” for another. Blind-tests can help sort out the specifics.
Blind-tastings are now an integral part of wine culture. The levelling conclusions so often drawn have hardly slowed the market for rare wines: Last year a bottle of 1811 Château d’Yquem sold for £75,000. Desire is a complicated thing. A mint-condition Stradivari set up by the Master himself would doubtless shatter auction records. As a concert instrument, it would be scarcely audible in larger venues, and would render most of the standard repertoire unplayable.
Old Italian sound is surely one of the great, ongoing constructs of the Western musical imagination. What began in Cremona has been built upon by generations of makers and restorers in response to generations of players, composers, and audiences. This experiment suggests that new violins can deliver the Old Italian sound in full measure – it’s just you may need to close your eyes to hear it.
Setting up a blind test.
Scientific testing has allowed dramatic improvements in the performance of everything from pharmaceuticals to loudspeakers. Why not violins? Blind-tests can provide makers with invaluable information about what players like and do not like about instruments. It can provide players with a bias-free zone in which the intrinsic qualities of an instrument come sharply into focus. Though a full double-blind study is difficult to organize, much can be learned by getting together with some colleagues, some interesting instruments or bows, and a few pairs of welders’ goggles.
Choose goggles with the darkest possible lenses, then add some electrical tape along the bottom edge, as shown. To avoid any effects of presentation order, use an online random sequence generator, or simply throw a dice and ignore the repetitions. Testing listeners together with players provides additional information – and complications. To average out the vagaries of room acoustics, have listeners rotate positions. Find ways of reducing visual contact among them, lest they influence each other via body language. For small, informal studies, the results tend to speak for themselves, but can also be expressed in terms of percentages and averages, or turned into graphs. For larger studies, bring in someone with a background in statistics!
The Authors Reply
From the December 2012 issue of The Strad
We asked the authors to answer some of the most frequently expressed criticisms of their experiment.
‘The experiment is fundamentally flawed! The only plausible arbiter of violin sound is the listener, not the player.’
There are many plausible arbiters – listeners, players, ensemble leaders, recording engineers – and no a priori reason that any two should agree. Given that players alone can assess playability, and that players ultimately choose their own instruments, player-preferences seem a sensible place to begin.
‘Sample sizes were too small to draw conclusions.’
Sample sizes were adequate for the rather modest conclusions we did draw – but not for the sweeping generalizations expressed by outside commentators and media headlines!
‘Players can’t possibly judge an instrument’s projection.’
Agreed. But players do routinely estimate projection, and typically acknowledge (as many of ours did) the need to re-test in a hall with listeners. Remember though that this experiment deals with the subjective impressions of players, not the objective characteristics of instruments. Correlating the two is an important topic for future research.
‘You can’t judge an instrument in a minute, or even twenty minutes! It can take years to get the best out of a Stradivari!’
Or a new violin! Still, a player searching for a violin will routinely eliminate instruments after playing them briefly. Early impressions may not be definitive, but if players take them seriously, so too should researchers.
‘The Old Italians used were not the best of their kind.’
Best for whom? We would welcome the chance to redo the experiment using other examples. While individual preferences might well change, we doubt players would suddenly begin consistently distinguishing old from new.
‘The Old Italians were not optimally set-up or adjusted.’
This wasn’t the opinion of their owners, who asked us not to change anything. We checked regularly to see that nothing had changed.
‘There’s no such thing as an objective assessment of tone quality!’
Quality assessments are indeed subjective, but a scientific experiment that compiles and correlates many subjective assessments can yield objective information about what players tend to like or dislike.
‘Tests like this might work on wine, but violins have an astonishing number of variables.’
Tests like this work on humans. Ours was a first step, and a small one considering the depth and complexity of the subject. As progress depends on collaboration with players, makers, and collectors, we invite anyone interested to please contact the authors!