Month: August 2018

After the methods crisis, the theory crisis

This thread started by Ekaterina Damer has prompted many recommendations from psychologists on twitter.

Here are most of the recommendations, with their recommender in brackets. I haven’t read these, but wanted to collate them in one place. Comments are open if you have your own suggestions.

(Iris van Rooij)
“How does it work?” vs. “What are the laws?” Two conceptions of psychological explanation. Robert Cummins

(Ed Orehek)
Theory Construction in Social Personality Psychology: Personal Experiences and Lessons Learned: A Special Issue of Personality and Social Psychology Review

(Djouria Ghilani)
Personal Reflections on Theory and Psychology
Gerd Gigerenzer,

Selected Works of Barry N. Markovsky

(pretty much everyone, but Tal Yarkoni put it like this)
“Meehl said most of what there is to say about this”

  • Theory-testing in psychology and physics: A methodological paradox
  • Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it
  • Why summaries of research on psychological theories are often uninterpretable

    • (Which reminds me, PsychBrief has been reading Meehl and provides extensive summaries here: Paul Meehl on philosophy of science: video lectures and papers)

    (Burak Tunca)
    What Theory is Not by Robert I. Sutton & Barry M. Staw

    (Joshua Skewes)
    Valerie Gray Hardcastle’s “How to build a theory in cognitive science”.

    (Randy McCarthy)
    Chapter 1 of Gawronski, B., & Bodenhausen, G. V. (2015). Theory and explanation in social psychology. Guilford Publications.

    (Kimberly Quinn)
    McGuire, W. J. (1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual review of psychology, 48(1), 1-30.

    (Daniël Lakens)
    Jaccard, J., & Jacoby, J. (2010). Theory Construction and Model-building Skills: A Practical Guide for Social Scientists. Guilford Press.

    Fiedler, K. (2004). Tools, toys, truisms, and theories: Some thoughts on the creative cycle of theory formation. Personality and Social Psychology Review, 8(2), 123–131.

    (Tom Stafford)
    Roberts and Pashler (2000). How persuasive is a good fit? A comment on theory testing

    From the discussion it is clear that the theory crisis will be every bit as rich and full of dissent as the methods crisis.

    Updates 16 August 2018

    (Richard Prather)
    Simmering et al (2010). To Model or Not to Model? A Dialogue on the Role of Computational Modeling in Developmental Science

    (Brett Buttliere: we made a Facebook group to talk about theory)
    Psychological Theory Discussion Group

    (Eric Morris)
    Wilson, K. G. (2001). Some notes on theoretical constructs: types and validation from a contextual behavioral perspective

    (Michael P. Grosz)
    Theoretical Amnesia by Denny Borsboom

    (Ivan Grahek)
    Fiedler (2017). What Constitutes Strong Psychological Science? The (Neglected) Role of Diagnosticity and A Priori Theorizing

    (Iris van Rooij)
    More suggestions in these two theads (one, two)

    Open Science Essentials: Preprints

    Open science essentials in 2 minutes, part 4

    Before a research article is published in a journal you can make it freely available for anyone to read. You could do this on your own website, but you can also do it on a preprint server, such as psyarxiv.com, where other researchers also share their preprints, which is supported by the OSF so will be around for a while, and which allows you to find others’ research easily.

    Preprint servers have been used for decades in physics, but are now becoming more common across academia. Preprints allow rapid dissemination of your research, which is especially important for early career researchers. Preprints can be cited and indexing services like Google Scholar will join your preprint citations with the record of your eventual journal publication.

    Preprints also mean that work can be reviewed (and errors-caught) before final publication.

    What happens when my paper is published?

    Your work is still available in preprint form, which means that there is a non-paywalled version and so more people will read and cite it. If you upload a version of the manuscript after it has been accepted for publication that is called a post-print.

    What about copyright?

    Mostly journals own the formatted, typeset version of your published manuscript. This is why you often aren’t allowed to upload the PDF of this to your own website or a preprint server, but there’s nothing stopping you uploading a version with the same text (so the formatting will be different, but the information is the same).

    Will journals refuse my paper if it is already “published” via a preprint?

    Most journals allow, or even encourage preprints. A diminishing minority don’t. If you’re interested you can search for specific journal policies here.

    Will I get scooped?

    Preprints allow you to timestamp your work before publication, so they can act to establish priority on a findings which is protection against being scooped. Of course, if you have a project where you don’t want to let anyone know you are working in that area until you’re published, preprints may not be suitable.

    When should I upload a preprint?

    Upload a preprint at the point of submission to a journal, and for each further submission and upon acceptance (making it a postprint).

    What’s to stop people uploading rubbish to a preprint server?

    There’s nothing to stop this, but since your reputation for doing quality work is one of the most important things a scholar has I don’t recommend it.

    Useful links:

    Part of a series:

    1. Pre-registration
    2. The Open Science Framework
    3. Reproducibility