Leadership group name:

Data and Evidence Collection Leadership Group

Leadership group members:

Ruiming Cash, Carla Davison, Katrina Sire, Deirdre Frank, Jenny Liou

Describe your leadership group purpose/aim:

The purpose of our leadership group was to collect and analyze evidence and data from ACI participants to make sense of our ACI work:  its impact on us as instructors, its impact on our assessment practices, its impact on students, and its potential impact going forward. In particular, we hoped to determine how the ACI impacted our students’ experience of English 101 and the completion rate of students who come from historically underrepresented groups.

Given the time and resources available to us, we determined that the best way to gather and analyze data was to survey ACI participants and invite them to help us analyze the data they provided. We designed an activity for the April 2023 convening in which ACI participants responded individually to a Google Forms survey that asked about their experiences with labor based grading, the efficacy and outcomes of their labor based grading instruction, student experiences based on the feedback they have received from students, and their experience of the ACI. Participants were then asked to discuss their responses in breakout groups and to post their collective analyses in Jamboard.

What did you accomplish?

The Google Forms/survey data provided by ACI participants was the most valuable data we collected. The Jamboard responses were not as robust as we hoped. From the Google Forms survey, we were able to accomplish the following:

• Descriptive summary/analysis of ACI participants’ instructional methods and how they have changed as a result of ACI
• Descriptive summary/analysis of ACI participants’ assessment methods and how they have changed as a result of ACI
• Descriptive summary/analysis of ACI participants’ reports of student feedback/experiences and how they have changed as a result of ACI
• Quantitative analysis of challenges ACI participants face and descriptive summary/analysis of these challenges
• Descriptive analysis of how ACI participants’ practices are or have become more antiracist and equity based as a result of ACI
• Coding of data to determine patterns within ACI participants’ pedagogy, practice, and philosophy
• Report detailing findings above
• Plan for future work based on limitations of this study

Include an artifact related to your work.

• ACI participant survey April 2022 (raw data)
• Jamboard data (raw)
• Summary of survey data/responses
• Presentation from June convening
• Report/analysis of survey results

What could be the next step to continue the work?

As part of the ACI Leadership Group tasked with data and evidence collection, we hoped to collect data from students. We recognize that student voice is perhaps the most important voice to consider when both implementing alternative assessment practices in the classroom and determining the overall impact of these assessment practices. One shortcoming we identified in our data collection for the ACI project was the lack of time for IRB approval, which we determined to be required for collection of student data (artifacts, journal responses, surveys, etc.). As a result, we recommend that future grants/projects begin with IRB proposals so that student data may be collected and so that student voice is represented.

In order to make the IRB process easier for future grant participants and/or any faculty member or student wishing to gain IRB approval, we propose creating a template for IRB submissions that can be used at institutions across the state. Because there is no state-wide IRB process, and because some institutions do not have IRB processes, we plan to research IRB approval processes across the state and create materials that can be helpful to faculty members from any institution. We will create templates that provide language to assist with describing the scope of antiracist/equitable assessment projects, alternative assessment practices, descriptions of benefits and risks of these projects, and the need for student feedback/student voice. We will also create a document that details “best practices” for those seeking IRB approval. We hope to deliver these templates/documents as a guide for faculty and students seeking IRB approval. We also hope that our work helps establish a statewide network for researchers from community and technical colleges, including students. This could allow a better pathway to publication for faculty and student researchers alike. We hope to collaborate with other groups who are working toward similar goals and initiatives (for example, WA CURE).