How Not to How Not to Analyze Data: John W. Tukey Against the Mechanization of Statistical Inference, Alexander Campolo.

Password: Evidence2020

• Post-WWII / late 1900s: sciences defined knowledge as “the inference from sample to population”
  • Evidence evaluated by null hypothesis testing and significance at the 5% level
    
  • Tangential: many criticisms of threshold (arbitrary, artificial “straw-person” in null hypothesis, based on incoherent synthesis of different methods, causes estrangement from effect size and statistical power)
• How did statistical tests become ossified into mechanical, context-independent procedures?
  • Porter shows behavioral sciences need to initiate new members and defend against external critics
  • Tukey did not have this problem – he was a singular elite
• Tukey evokes values underlying rationale behind use of tests
  • Did not envy physics, did not want the reductionist “mathematization of nature”, did not want extension of Weberian rationality
  • Wanted judgement, experience and even pluralism
• Tukey advocated for empiricism based in perception and sense
  • Manifesto called Data Analysis and Behavioral Science or Learning to Bear the Quantitative Man’s Burden by Shunning Badmandments
  • Three criticisms of significance tests: sanctify results, reduce to binary decision (signal vs. noise), crowd out other analytical tools
• “Probabilistic worlds require both techniques for producing knowledge under conditions of uncertainty and, as Theodore Porter has shown, institutions for producing trust, perhaps even faith, in such knowledge.”
• “…subjective judgement, driven by empirical perceptions, must shepherd quantification across threacherous spans”
• Tukey was friends with Claude Shannon
• Tukey gave data an agentive voice
  • “After the single-minded pursuit of optimality, “the next fetish to be attacked,” he suggested with some relish, “is the fetish of objectivity,” understood as “the fallacy that to a single body of data there corresponds a unique appropriate analysis.”
  • Not procedural, mechanical, optimal valuation of objectivity but a communicative raitonality
    • Orit Halpern vs. Daston, Galison, and Porter
• Tukey had a historical understanding of science
  • Urged behavioral sciences to look to early days of elder sciences - Still problems with Tukey: ideas provide little consensus, connected to him, reflect an elite sensibility (mechanical forms of inference do not have this problem)
• Produces a “scientific self” from Gallison and Daston