Abstract

Upon pausing to consider the progress accomplished in the implementation of a revised general education (GenEd) program, faculty, and staff of the General Education Assessment Committee (GEAC) at Virginia Commonwealth University identified key lessons that supported success and opportunities to enhance assessment in our program. This article shares what was learned. The experience underscored the importance of holistic design and the need for stakeholder engagement, clear communication, and administrative support. Further, multidisciplinary faculty engagement emerged as a pivotal component for enhancing assessment quality. These insights offer a concise blueprint for faculty-led GenEd assessments, highlighting the importance of inclusivity and collaboration in higher education practices.

Introduction
To set the stage for the General Education assessment that Virginia Commonwealth University (VCU) embarked upon in enhancing its general education curriculum, it is essential to trace back to the origins of this endeavor. The process of completely revising the general education curriculum at VCU began in January 2016 when the Provost issued a charge to the GenEd Task Force. Unfortunately, this charge explicitly declared that “Assessment matters, or the development of student learning outcomes, . . . [are] out of scope for the GenEd Task Force.” The Task Force delivered their report in the spring of 2018, and the General Education Curriculum Committee (GECC) was convened in the fall of 2018 with the charge to implement and maintain the general education curriculum for the university. The general education program at VCU has been branded as “ConnectED” ever since.

Optimally, curriculum and assessment are inextricably linked, informing each other, in order to develop a comprehensive and strongly aligned GenEd program. However, a separate workgroup of faculty and assessment coordinators began working independently of the GECC on broad learning goals and outcomes, and an official GenEd assessment committee was not convened until the fall of 2019. The General Education Assessment Committee (GEAC), composed of faculty representatives from every academic unit offering general education courses, ensured all stakeholders had a voice. Notable features include the work led by faculty representing all units at VCU, and the necessity for strong cooperation between curriculum and assessment to better align learning goals and outcomes. These features became increasingly important as the GEAC focused on researching best practices for assessment across various universities, informed by research (e.g. Banta and Palomba, 2015; Leskes and Wright, 2005; Suskie, 2018), conversations with peer institutions, and engagement with professional organizations including The American Association of Colleges and Universities (AAC&U) and The Association of General and Liberal Studies (AGLS). Since that time, the faculty-led GEAC developed and implemented a GenEd assessment plan, yielding findings that have advanced our understanding of student learning at VCU. We also learned about processes that work and those that need to be revised as we move forward.

Reflecting on our work in developing and implementing a general education assessment plan, we offer two key lessons: 1) holistic design is the foundation for an effective assessment process, and 2) multidisciplinary faculty engagement enhances assessment quality and continuous improvement (see Figure 1).

Figure 1. GenEd Assessment Framework

Figure 1 illustrates the holistic assessment design and multidisciplinary faculty engagement embedded in and contributing to effective GenEd assessment. The holistic assessment design showcases the segments that lay the foundation for an effective general education assessment. As the main pillars, the multidisciplinary faculty engagement components highlight the interplay of faculty dialogue and process through assignment scoring events and ConnectED Conversations. These elements are organized using common language, curriculum maps for assessments, and Taskstream OAP for multidisciplinary data collection. The below lessons will also expand on effective GenEd assessment.

Lesson 1: Holistic design is the foundation for an effective assessment process.

The first lesson learned from the implementation of VCU ConnectED revolves around the importance of a holistic design approach to GenEd (see Figure 1). Holistic design in GenEd is about creating a cohesive and integrated educational framework and engaging all key stakeholders. It involves meticulously weaving together various elements of the assessment ecosystem from the beginning and ensures that each part works harmoniously towards common goals. Contrary to public belief, assessment transcends mere summative assignment grading and should permeate throughout teaching and learning in general education courses. Under a holistic model, assessment is present at the onset of GenEd in the articulation of learning outcomes. It informs curriculum and classroom pedagogy and extends through the design of related assignments and rubrics. It also encompasses the spectrum from faculty communication to student outreach. Without processes and procedures to integrate assessment plans into the curriculum from the start, the GenEd curriculum, and assessment thereof, ended up being developed separately. At VCU, the disconnection between the GenEd curriculum and assessments led to weak alignment between select GenEd courses and GenEd learning outcomes, posing challenges in course adaptation to meet assessment requirements. Further, faculty teaching GenEd courses were, at times, unaware of the learning outcomes aligned to their courses and would have to try to retroactively fit GenEd assessment requirements into the established coursework. Consequently, assessment was then perceived as an addition rather than an integral part of course design.

Early stakeholder identification and engagement
To remedy these challenges, GEAC recognizes the role of early stakeholder identification and engagement in holistic design. The first step in the holistic design of GenEd is the identification and invitation of all stakeholders while providing these individuals with clear role expectations for the program (International Association for Public Participation, 2007). Stakeholders with both positional power in the program (faculty) as well as those impacted by the program (students) need to be included (Freeman, 1984; Helbig et al., 2015). Early engagement and role clarity support stakeholder commitment to the work and can foster relational trust, which is necessary for effective data-informed decision-making (Schneider & Bryk, 2002). In hindsight, our general education curriculum (GECC) and assessment committees (GEAC), although representative of faculty across disciplines, left a seat unfilled to collaboratively include the student voice in the development of GenEd. Alongside stakeholder inclusion, regular and effective communication with these stakeholders is equally essential. Our communications plan originally included periodic email updates, information sessions, and scoring events for faculty teaching general education courses. Yet, despite these efforts, the pilot assessment plan initially did not specify a clear avenue for sharing data with our stakeholders. These stakeholders encompass internal members, such as faculty, staff, and students, and external entities like families, employers, and communities. An incomplete stakeholder group led to incomplete communications and highlighted the need for comprehensive communication strategies that engage various stakeholders in active participation through ongoing dialogue, a fundamental principle of holistic design. As one of the efforts to engage stakeholders, the GenEd website has now included content and design features to communicate with multiple stakeholder groups, both internal and external to the university.

Multi-level Administrative Support
Holistic design in GenEd also requires integrated support at various institutional levels. For institutions where GenEd efforts are predominantly faculty-led, like VCU, backing from upper-level administration is a prerequisite. Data reporting and communication of findings, typically a task for institutional administrative units, was an area our institution initially overlooked. Centralized staff support, crucial for executing the ’closing-the-loop’ step in assessment processes, was absent in our early framework. This, in part, created a communication gap in select stakeholders like students and external parties. The good news: we learned quickly how important it is for university committees to have dedicated staff support and clear roles for administration and faculty. Our institution allocated both staff and administrative leadership efforts to ensure systematic sharing of data and information both internally and externally. Such support lends credibility to GenEd assessment initiatives, benefiting both students and faculty. Now, GenEd assessment is consistently addressed in high-level meetings involving associate deans from each college/school. Given the university’s large size, this method of cascading communication through deans to individual departments has proven effective, particularly considering departments’ tendency to respond more actively to directives from their own college’s dean rather than central administrators.

In the initial stages of our GenEd assessment efforts, VCU did not allocate administrative assistance, a gap we later identified. In conjunction with high-level support, we have learnt administrative assistance plays a vital role. Embedding administrative assistant roles from the design phase is strategic, significantly reducing the workload on faculty committee members. The administrative duties encompass detailed data collection and sophisticated statistical analysis, which are fundamental for making informed decisions. Administrative roles also manage meeting and event planning, allowing committee members to concentrate their expertise on core tasks and strategic discussions. Therefore, administrative support streamlines GenEd assessment operations and enhances both its quality and effectiveness.

Procedures for Curriculum and Assessment Collaboration
Effective structural workflow management is paramount for GenEd programs in a faculty-led environment. Articulating the structure and accompanying procedures by which general education student assessment data is reported, analyzed, and used for continuous improvement is essential to sustainable and meaningful stakeholder engagement.

Codification of the assessment workflow supports transparency and shared understanding where both can positively impact knowledge sharing broadly in an institution as well as build relational trust amongst those engaged and impacted by GenEd curriculum and assessment policy and outcomes (Bryk et al., 2015; Lechner & Floyd, 2007). At VCU, establishing both GECC and GEAC with bylaws, policies, and procedures provided for clearly defined and broad engagement of faculty with clear roles and responsibilities within the committees. However, it proved insufficient to guide the work across these two committees. After collecting, evaluating, and aggregating student work samples and outcomes from our initial assessment cycles, we identified a lack of structure, or collaborative space, between the two committees, to support the shared review of data.

This gap hindered the joint review and discussion of the collected outcomes data. The GEAC identified opportunities to refine learning outcomes criteria, improve assignment design, and streamline curriculum alignments. However, an operational process was needed

between the two committees to convey these assessment insights to the GECC to enhance curriculum and subsequently learning. In retrospect, our committees would have benefited from a clear statement of procedure for data review – on what cycle and by whom – and what the process would be for data-informed interpretations to fold into curriculum design. This experience underscores a broader challenge in higher education, emphasizing the importance of establishing clear procedural links for interdisciplinary collaboration as a cornerstone of holistic design.

VCU ConnectED’s first lesson underscores the indispensability of holistic design in GenEd assessment. This design, encompassing stakeholder engagement, clear communication, administrative support, and codified procedures for collaboration is foundational for effectively implementing and improving GenEd. The holistic design ensures that every stage of GenEd, from curriculum planning to assessment execution, is interwoven with stakeholder input and administrative backing. This insight into holistic design serves as a valuable blueprint for other institutions seeking to establish or refine their GenEd assessment processes

Lesson 2: Multidisciplinary faculty engagement enhances assessment quality and continuous improvement.

Multidisciplinary Integration
If the holistic design forms the foundation of GenEd assessment, then multidisciplinary faculty engagement is undoubtedly its main frame (see Figure 1). Building on the initial research (Hutchings, 2010; Suskie, 2018), the GEAC identified three areas for focused development of faculty engagement. The first focus was on faculty communication using a common language, aimed at improving how faculty from different disciplines work together in the assessment process. Standardizing assessment terminology was crucial in a multidisciplinary context to ensure that all faculty members, regardless of their academic background, had a shared understanding of the assessment process. This standardization not only bridged disciplinary gaps but also illuminated the unique contributions each discipline could offer. For instance, in developing rubrics for quantitative literacy, faculty from mathematics initially drafted technical criteria. Subsequently, faculty from non-math backgrounds contributed, refining the language to make it accessible and comprehensible to a non-technical audience. This collaborative effort exemplified how varied academic perspectives could enrich the assessment tools, making them more inclusive and representative of the diverse academic community at VCU.

Assessment Plan Development
The second focused effort involved a structured approach for developing an evidence-based assessment plan. Curriculum maps detailed the alignment of courses and learning goals, which provided the GEAC a clear plan of which courses to collect data from in each assessment cycle. GEAC prioritized student work collection from all participating GenEd courses. Scoring of these samples provided learning outcomes data across disciplines, which was vital for a comprehensive analysis of student performance and learning patterns.

Results Documentation
In this effort, the adoption of Taskstream OAP, formerly known as AQUA, proved to be instrumental. It simplified the systematic collection and analysis of data across disciplines. These results helped pinpoint areas needing improvement and highlighted successes in the GenEd curriculum and assessment. This laid a foundation for a granulated, data-informed programmatic level assessment for GenEd. Lastly, these efforts centered on documenting and communicating assessment results effectively with institutional stakeholders such as The State Council of Higher Education for Virginia (SCHEV) and The Southern Association of Colleges and Schools Commission on Colleges (SACSCOC).

GenEd Information Sessions and Faculty Scoring Events
With the goal of engaging all GenEd faculty from different disciplines, the GEAC organized “ConnectED Conversations” during each assessment cycle. These sessions were educational in nature, initially intended to troubleshoot and facilitate the process of collecting student work. As faculty mastered technical aspects, the sessions evolved to provide spaces for sharing ideas about assignment design and alignment. At the culmination of the year, to celebrate faculty work in GenEd, the GEAC and GECC collaborated in hosting an annual symposium. These half-day events featured guest speakers, panel discussions led by ConnectED faculty, and break-out discussion groups among stakeholders including faculty, administrators, and academic advisors. Pausing to recognize and express appreciation for the efforts of all ConnectED stakeholders ignited a surge of enthusiasm heading into the penultimate assessment activity, assignment scoring.
GEAC understood the importance of the input of a representative faculty while scoring the student work. For this reason, the GEAC created the summer scoring events. These events, where faculty evaluated student work and provided feedback on rubrics, became pivotal in facilitating cross-disciplinary dialogue and understanding. The scoring events involved scoring each student artifact by at least two faculty members from different disciplines, using rubrics that determined the alignment with the goals. Faculty participation in scoring events provided powerful experiences, fostering meaningful discussions about evidence of student learning across diverse disciplines. The data revealed statistically significant differences among faculty scores, highlighting diverse academic perspectives and revealing areas in the rubrics needing clarity and consensus. Particularly, the criteria with the highest variability in scoring were those receiving the most suggestions for improvement in written faculty feedback provided during group calibration, which indicated the need for clarity and modification. “ConnectED Conversations” served as platforms to share these emerging findings during subsequent assessment cycles. Faculty responded positively to the acknowledgment of their efforts. Insights drawn from a blend of disciplinary viewpoints were pivotal in refining the rubrics for future assessment cycles.

This process exemplified the value of incorporating diverse faculty expertise in enhancing
GenEd assessment tools and demonstrated how multidisciplinary engagement can lead to more effective GenEd assessments.

The GEAC’s journey emphasizes the strategic value of multidisciplinary faculty engagement in fostering a culture of assessment. By incorporating discipline-specific samples and encouraging broad-scale faculty engagement, GEAC effectively harnessed diverse academic insights across academic disciplines. Learning goals and assessment outcomes provided the shared language to facilitate meaningful conversations about student learning. Emerging data and interpretations point to opportunities to collaboratively improve curriculum, assignment design, and assessment rubrics with the intention to continuously improve our approach to understanding student outcomes and our students’ learning. Ongoing multi-disciplinary collaboration is crucial for successful assessment practice in higher education.

Conclusion
In conclusion, GEAC’s experience offers two valuable lessons for effective GenEd assessment. Holistic design emerges as a critical foundation for GenEd assessment, which demands comprehensive stakeholder engagement, multi-level administrative support, and effective collaborative procedural links. Concurrently, the engagement of multidisciplinary faculty constitutes the core of the main pillars in enhancing GenEd assessment’s depth and breadth. Presented in Figure 1, the foundational principles serve as a strategic framework and provide a blueprint applicable to other institutions. Such a blueprint is especially beneficial for those institutions with faculty-led GenEd programs. Implementing this model will lead to richer, more comprehensive assessments that resonate more closely with the unique needs and perspectives of diverse academic communities.

Author

Dr. Stephanie Cai serves as the Undergraduate Assessment Coordinator for the Provost’s Office at Virginia Commonwealth University (VCU), where she specializes in the in-depth analysis of student assessment data. With a robust background in Economics, she earned her PhD in Economics from Lancaster University, England. At VCU, she is instrumental in both authoring GenEd assessment reports and creating comprehensive dashboards that illuminate assessment strategies and outcomes. Her collaboration extends to the General Education Committee and the Quality Enhancement Plan (QEP) team. Her expertise in economic models and data analysis significantly enhances VCU’s educational reporting. She provides detailed insights and practical recommendations, fostering continuous improvement across the university.

Ms. Jeen Joy is a third-year doctoral student in the VCU Foundations of Education Department. Currently, she works as a research assistant with the Foundations Department and the Institute of Collaborative Research and Evaluation (ICRE). Her work revolves around quantitative data analysis, data modeling, evaluations, and other data analysis. She has experience working on multiple research projects that aim to improve teachers’ assessment practices. She also worked with the General Education Assessment Committee in collection, cleaning, and analysis. Her work provided insights into understanding the different challenges the assessors faced in adhering to the university standards. Her work was useful in creating guidelines for professors to improve their assessment practices.

Ms. Virginia Wray Totaro is an Associate Professor in the Department of Focused Inquiry, the unit that houses first-year academic seminar courses for ConnectED. She also serves as the Assessment Coordinator for University College and co-chair of VCU’s General Education Assessment Committee. With a background in sociology and a passion for transformative teaching and learning, adding the role of assessment coordinator in 2015 was a natural step. Virginia joined the Gen Ed assessment workgroup in 2018 and is certified as a Watermark Outcomes Assessment administrator.

Dr. Christy Tyndall serves as Faculty Director for General Education at Virginia Commonwealth University. She and her co-director work with administrators and faculty to promote the advancement of student learning through ConnectED, VCU’s General Education Program. Christy has played an active role in leading general education assessment efforts at VCU since 2019. Also a faculty member and Student Wellbeing Coordinator in VCU’s Honors College, Christy develops and implements innovative programming to promote wellbeing, mindfulness, and community-building. A life-long student of psychology and educator for 20 years, she teaches courses in applied positive psychology, social psychology, and qualitative research methods. Christy’s areas of expertise include working with others to identify personally effective strategies for promoting holistic wellbeing, working through cognitive barriers, motivation, and teaching/learning in higher education.

Dr. Angela Wetzel currently serves as the director of data analytics and as an affiliate faculty member in the Foundations of Education department of the Virginia Commonwealth University School of Education. In the School of Education, Wetzel provides leadership for a complex and diverse set of activities designed to provide actionable data analytics to support decision making related to strategic planning, program quality and productivity, and student learning. Angie also serves on the VCU General Education Assessment Committee leadership team. In this capacity, Angie supports faculty communication and calibration and scoring; Angie also provides general education assessment analytics to guide improvements in the articulation of student learning outcomes, assignment alignment, and rubric design.

Dr. Maria Carolina Yaber is an Associate Professor in the Department of Biology and Co-Director of the General Education Program. She earned a B.S. in the Biology Department at the Universidad Simon Bolivar – Venezuela. Later she earned a PhD. in the Department of Biological Sciences at Purdue University in Indiana. Her graduate research training was in Ecology and Evolution, and Population Biology. Dr. Yaber has taught at Hampden Sydney College, James Madison University, and Richard Bland College. She started teaching at Virginia Commonwealth University in 2015, where she has taught a number of courses including: Physiology, Basic Human Anatomy-Lecture and Laboratory, Tropical Biology, Introductory Biology, Comparative Vertebrate Anatomy. Dr Yaber has served in many committees, including the Biology Diversity Equity and Inclusion Committee, Faculty Council (secretary), Biology Undergraduate Committee (chair), General Education Assessment Committee (chair).

References

Banta, T. W., & Palomba, C. A. (2015). Assessment essentials: Planning, implementing, and improving assessment in higher education (2nd ed.). Jossey-Bass.

Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard Education Press.

Freeman, R. E. (1984). Strategic management: A stakeholder approach. Pitman.

Helbig, N., Dawes, S., Dzhusupova, Z., Klievink, B., & Mkude, C. (2015). Stakeholder engagement in policy development: Observations and lessons from international experience. In M. Janssen,

M. Wimmer, & A. Deljoo (Eds.), Policy practice and digital science (Vol. 10). Springer.

Hutchings, P. (2010). Opening doors to faculty involvement in assessment.

International Association for Public Participation. (2007). Iap2 spectrum of public participation. Retrieved October 30, 2023, from http://www.iap2.org

Lechner, C., & Floyd, S. (2007). Searching, processing, codifying, and practicing: Key learning activities in exploratory initiatives. Long Range Planning, 40, 9–29.

Leskes, A., & Wright, B. (2005). The art and science of assessing general education outcomes. AAC&U.

Schneider, B., & Bryk, A. (2002). Trust in schools. Russell Sage Foundation.

Suskie, L. (2018). Assessing student learning: A common sense guide (3rd ed.). Jossey-Bass.