Tonight I watched a JMBE Live! session with the authors of the article “From Novice to Expert: An Assessment to Measure Strategies Students Implement While Learning to Read Primary Scientific Literature.” The session was presented on May 19, 2023 by Melissa McCartney, Min Zhong, Cerrone Foster, and Miriam Segura. The group worked together to study the analysis of primary scientific literature (PSL). Segura asked the attendees: do you use PSL in your classes and how often do you use PSL in your classes? Segura reviewed what is known about how students and faculty read scientific articles. In 2019, Segura and team published in LSE-CBE a study with 11 students and 6 faculty who read PSL aloud voicing thoughts. The sessions were recorded, transcribed, and coded. The three themes were: thinking tools, scientific literacy and process skills, and comprehension difficulties. The tools used were re-reading, summarizing, using a reference point/prior knowledge, underlining, taking notes, and replying on definition provided. McCartney explained the research question they had for their study: can we create a tool for measuring primary literature? For this study, they developed Likert scale questions and conducted a pre-survey in courses at their institutions. The first author of the study was the graduate student who performed EFA analysis. Zhong described their implementation of modules to improve reading of PSL. The approach used a blank PDF vs an annotated version, an outline assignment, and an eight question reading quiz with three content questions and five scientific literacy questions. At both Auburn and East Tennessee State, the course included teaching or peer learning assistants. Foster described the results of the pre-test, post-test, and change. Each campus had a slightly different implementation, and there were some differences in the changes (pre/post) observed. In a subsequent study, the team used Science in the Classroom and found the annotations to be helpful. In this subsequent study also published in JMBE, the research team performed a qualitative assessment of behavioral engagement in using annotations. Foster noted that there was some evidence to support how the use of annotations may help. The PSL reading strategies assessment can be used as a reliable and validated diagnostic tool for the development of science literacy skills. The team also concluded that reading the PDF alone does little to promote development of reading strategies in students learning to read PSL. Annotated papers did not quantitatively influence in specific reading development for PSL but did help break down PSL into manageable pieces. Maloy asked about the statistical analyses: EFA, exploratory factor analysis and CFA, confirmatory factor analysis. McCartney described Science in the Classroom platform and annotations. Foster explained the pre/post tools used. Segura noted that for earlier students (first and second year, for example) more scaffolding is needed. Zhong emphasized how the number of variables and importance of connections to course topic and interests. Foster explained that the peer learning assistants facilitated the discussion with students. This session was packed with research questions and studies!
