Valuing Scholarship, Methodology, and Leaders in Changing Scholarly Landscapes

A discussion about how to place value on scholarship is also a conversation about the value of scholarship and how to produce good scholarship.  The best scholarship is that which communicates to educators, researchers, and students information that makes a contribution to the field.  This requires language that has meaning, and often language that is more effectual because it is established by the community.  This also requires peer review that questions methodology and is established in an extensive understanding of the subject matter, in order to evaluate quality of literature review.  It also requires researchers to completing their research in a way that allows for replicability.  All of this is independent of the funding model of the journal – though who has the knowledge requirements I mentioned above and also wants to dedicate time and energy to this work if peer review and journal editing are not part of tenure considerations?

I think an interesting element to this conversation is that academics might be padding their resumes by publication in predatory journals, as Ben mentions.  Academics are so invested in publication, in part because of tenure requirements, that they are making various research sacrifices in order to have an interesting article that is able to be published.  In 2012, confronted with a crisis in research replicability, psychologists suggested the following to increase the reliability of their research:

…using undergraduate projects as a route for replicating existing studies; encouraging adversarial collaborations where sceptics replicate studies alongside original investigators; providing accessible outlets for publishing replications; opening up data, methods and workflow; and pre-registering studies including all the intended methods and analyses. (Owens, 2012)

Note these suggestions do not include strengthening the peer review process to keep out shoddy articles or ruining the reputation of the journals that published these articles.  Instead, opening data and methodologies is suggested as a way to increase quality scholarship.

In May 2017 I heard a panel discussion on BBC’s Newshour Extra titled, “What’s Wrong with Science?” that included academics and journal editors calling for a change in the way that researchers submit to academic journals.  Instead of submitting their article after the research is complete, this panel suggested that methodology be submitted for acceptance before the research is done.  The research would be accepted based on its methodological merit, not on the results.

Karen notes that open access journals provide an opportunity to publish failed research.  In a way, the fact that for-profit journals are not publishing this kind of information is evidence of a flawed system; one that researchers are already invested in participation.  Reforming the real-world scientific process through Team Science, as discussed by Patrick, is a great idea.  Academics, or administrators in higher education, are setting the standards for what we value from each other and, also, how we value it.  It’s a cycle that could use some reformation, which is an interesting to think about on this 500th anniversary of the Protestant Reformation.  While we remember Luther (in all of his mythical splendor), we might also remember Aaron Swartz and Alexandra Elbakyan – leaders in the open scholarship movement.

This post was written for a faculty learning community on open scholarship.  It was originally posted on 10/31/17 on the #openberg website.

Author: Jess Denke

Assessment and Outreach Librarian at Muhlenberg College

One thought on “Valuing Scholarship, Methodology, and Leaders in Changing Scholarly Landscapes”

  1. There is another interesting podcast here: It features a UVA psychologist, Brian Nosek, that studied reproducible results in experiments. He found that out of 100 repeated experiments, well less than half confirmed the original results. The podcast is about innocent cognitive biases in favor of positive exciting results over negative or boring results. But these biases can be overcome through better method.

    This is a good general interest article from 2010 on bad medical science from The Atlantic:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: