@Freese

() - Jeremy Freese

Journal:
Link::
DOI::
Links::
Tags:: #paper #Methods #OpenScience #General
Cite Key:: [@Freese]

Abstract

Computing and programming advances continue to dramatically increase the scale and sophistication of analyses that quantitative social science conducts. One consequence of this increased complexity is that quantitative research articles are often required by space and stylistic constraints to “paraphrase” or omit discussion of many particulars of decisions about constructing variables, excluding observations, and specifying models (among other things) that are made in the course of research. This, in turn, has prompted increased concern about the latitude for published results to reflect a series of analytic decisions whose implications or even existence are underdocumented in the article, especially as these decisions might, for whatever reason, mostly happen to favor the author’s argument (e.g., Peng, Dominici, and Zeger 2006; Ho, Imai, King, and Stuart 2005). Underdocumented analyses also make it harder to follow the analytic decisions of others in trying to build off and elaborate past research. In other words, the interests of quantitative social science are best served by maximizing the transparency of analytic work—the extensiveness and precision of information available about how published results were derived from data—but the increasing complexity of analytic work makes printed journal space ever more inadequate for providing such detail.

Notes

“Increased replication standards would be beneficial for the credibility of sociological research because it increases confidence that work can be replicated, but they are also valuable because they make published work more available to elaboration and extension by others and they afford the best opportunity for exemplary work to contribute to teaching other members of the profession” (Freese, p. 6)

“the main instances in which depositing code would imply more work would be precisely those instances in which more work is desirable anyway, in the sense of ensuring the integrity of the “chain of evidence” (King 2003: 100)” (Freese, p. 7)

“Important to recognize is that no argument against making data available is a good argument against providing explicit information about data availability. Researchers could state “Data will be shared with individual investigators for verification purposes only” or “Due to confidentiality restrictions, data cannot be shared with others even for purposes of verification.”” (Freese, p. 9)

“Regardless, it is false to conclude that depositing code at the time of publication is only worthwhile if data are also available” (Freese, p. 10)