Critical Reading

This week I set up to improve my critical reading skills. I felt like I was not absorbing much from the papers I was reading and having a hard time identifying the key points, goals and potential caveats of a study. I mean, I still think I am having a hard time with all that, I don’t think my one-week effort solved it perfectly – but it did help. Long story short: have questions in mind going in, take notes and be prepared to spend a few hours.

It is important to keep in mind that reading an article from beginning to end like a book is not advisable by any of the sources I read. I patched a bunch of ideas from different websites (links at the end) and created a set of questions to guide me throughout the reading as I take notes. Here, I will comment on them in order.

First, before starting, the question I need to answer is “why am I reading this?” – that is an important question to set my mindset as I go through the rest of the activity. Interestingly, I answer that even before deciding what I will read, as it will guide my decision making about which paper I will choose. Which is the second task: what to read? I know what I need to get out of the reading from the first question and that should be enough to make me decide if I will search for reviews or original articles or something else. Then, I browse through titles and open a few potentially interesting articles and go check their abstracts briefly. That should be enough to have a good sense of whether it is worth reading for my purpose.

With an article chosen, I am still not committed to it, but now it is time to start taking notes. The first step of notes and question aim to get an overview of the article and will be taken more systematically. In a notebook, I will write the basic information about the article: citation (authors, title, year), where the authors are from to start developing a sense about which research groups (and where) are doing what. I would also take note on the kind of paper (review/original research). Then I would write the answer to my first question “why am I reading this?” 

The set of questions I will try to answer from the abstract and potentially from reading the figures:

• What are the major ideas?
• Why was the study made? How was it made?
• What are the findings and what is the kind of data/evidence supporting it?
• Where was it done/who are the subjects?

This should have summarized the major points and told me whether my goals for reading the paper will be achieved. For now, I am still playing around with the order in which to read the section of the article. I will try ADIRM for now: Abstract, Discussion, Introduction, Results, Methods. I will also be taking notes on which reading strategy I use. 

This is the questions I will keep in mind for the discussion:

• Were the results interpreted appropriately?
• Are there other ways to interpret the results?
• Have the methods been discussed?
• Are the theoretical claims justified? 
• Are the hypotheses addressed? Have they been rejected or supported?
• What are the implications of the findings?
• What suggestions are made about future research?
• Are there any good ideas that could have other applications or extensions? Can they be generalized further? Any possible improvements that might make important practical differences?

Some might be challenging with the ADIRM structure, but if the session is well written it should be fine. It is advisable to take a look at the discussion again at the end of reading the paper. The last question will be repeated often, it is part of the idea of not only developing a critical reading but a creative reading – trying to get inspired by the text to go beyond what is being discussed. This could be interesting to start thinking more critically and creatively about science as a whole. It is hard, though. I need to try it again, I have applied this method to a paper about reading (it was not a research article, more an opinion/thought exercise) and the most creative I got was the idea to put together a set of questions from different sources to make my own guide – hardly could be called creative. I also applied to a pivotal paper that is the base of my current project, so I guess the creative side of it had already been thought and designed to create my project. I will try again and comment more on a future post.

For the introduction, these are my questions:

• What is the purpose of the paper?
• How is the story going to be told? (skim over pictures if needed)
• Why is it relevant?
• What is already known about the topic?
• What are the knowledge gaps?
• How will it address the gaps?
• What specific predictions or hypotheses are being tested?
• Are there any good ideas that could have other applications or extensions? Can they be generalized further? Any possible improvements that might make important practical differences?

I like reading introductions, it always felt like the more passive part of the paper just being fed the information. Applying these questions has changed that…

For the results and methods these are my questions:

Results

• Do the results support the predictions/hypotheses?
• Are the results reported unbiasedly?
• Are there any good ideas that could have other applications or extensions? Can they be generalized further? Any possible improvements that might make important practical differences?

Methods

• What research techniques are used?
• Are the methods a valid test of the predictions/hypotheses?
• Has any confounding or extraneous variable been overlooked?
• How do the methods compare to the methods from similar studies?
•  Are there any good ideas that could have other applications or extensions? Can they be generalized further? Any possible improvements that might make important practical differences?

Not much to comment about these questions, the results ones could be refined I think, they don’t guide me much. To finalize, I take another look at the discussion and at my answers and go to my final questions:

• 1 to 3 sentence summary:
• Citations in the paper that I should read in the future
• If you were going to start doing research from this paper, what would be the next thing you would do?

That is it. Think all the time, use the reading as a thinking exercise!

Below is the list of articles I read, in the order that I read them. My questions came from these sources, as did much of the reflection I made:

• Kresser Institute (geared towards health professionals, has two approaches to critical reading that could be used, mainly changing the order in which to read the sessions of an article)
• Entomology, Penn State program (most of my questions came from here, tailored to biology students and closer to my reality)
• Michael Mitzenmacher’s lab (Harvard Comp. Sci. prof) (emphasizes the importance of taking notes and brings up the importance of creative reading)
•  Art of reading a journal article: Methodically and effectively, by RV Subramanyam. (that is the article that I applied the first version of my questions. Emphasizes the importance of being systematic and was the inspiration for me to put together the list of questions instead of simply using the one from Penn State without making any additions)
• This is a cool Reading exercise for college students (I liked the teaching aspect here, might use one day)
• Purugganan & Hewitt – how to read a scientific paper (this article added the dimension of reflecting and asking questions before reading, the “why read this?” and “what to read?” I talked about at the beginning.