Wasana Karunarathne from The University of Melbourne presented (from across the world!) at the 2021 Lilly Conference online in May. The topic they presented was “Developing Generic Skills and Feedback Literacy Through Assessment.” Karunarathne talked about using “assessment for learning” using formative feedback and motivate learners to improve. Karunarathne wanted students to experience hands-on learning and use concepts/techniques to solve real world problems. Students gained experience with Excel to analyze and present data. I loved that they included communicating to a non-technical audience. Karunarathne also mentioned that the “freedom to investigate increases the level of competency.” I appreciate how Karunarathne spoke about improving collaborative skills while still allowing those who wish to work individually. Interestingly, their review sessions were mentioned as a way to help groups form and develop, and I was interested in how exactly this is dong. One important goal of the experience was to increase ability of learners to read, understand, and use feedback. Students received feedback and work on the same report to submit a final version later in the semester. Karunarathne explained how the pedagogical design consisted of a set of data related to business and economics and students chose the topic to write a report with conclusions/recommendations for a non-technical audience. To assess the impact of the pedagogical design, grades of the draft and the final reports were analyzed. They continue to analyze the data, including student feedback, and are currently mainly focused on quantitative methods.
The sample Karunarathne described consisted of 877 students. The average grade for the report, graded based on a rubric, increased from draft to final submission. Karunarathne also examined score improvements plotting boxplots of the change in assignment total scores pre-post feedback. Sudents found that the task was interesting with specific comments about how the Excel and analytical techniques were useful. Karunarathne also was transparent about the limitations of the study. Karunarathne concluded that the challenging problem and feedback helped them read, understand, and use the feedback to improve the task. This was labeled as “improved feedback literacy” that I had not heard before. Interestingly, improvements were not equal across the cohort. What can be done differently to reach more students? I find it useful to learn about the methods and experimental design others use in their educational studies to help inform what we can do differently.
