In Conversation with Michael Seelig, Ed.D, Editor of the Assessment Review

Dr. Natasha Jankowski is a higher education and assessment expert and the former Executive Director of the National Institute for Learning Outcomes Assessment (NILOA). She is co-author of the books Using Evidence of Student Learning to Improve Higher Education; the book Degrees that Matter: Moving Higher Education to a Learning Systems Paradigm, and the edited volume Student-Focused Learning and Assessment: Involving Students in the Learning Process in Higher Education. A forthcoming book with Stylus focuses on equity and assessment. Her areas of research include assignment design, transparency, evidence-based storytelling, equity, mapping, and alignment of learning. She holds a Ph.D. in Higher Education from the University of Illinois, an M.A. in Higher Education Administration from Kent State University, and a B.A. in philosophy from Illinois State University.

MS: You’ve been a nationally recognized leader in Assessment for many years.  What are some things you’re focused on right now during this unprecedented time?

NJ: First, congratulations on your new journal at CUNY and I’m honored to be interviewed!  As you know it’s been a challenging year in the assessment community with varied experiences and a lot of fatigue. Being a leader in this work, or leading a national organization at this time, is far more challenging when you can’t get on the ground to see what is going on and how people are faring.  Zoom is just no substitute for being with people to really think through and co-construct how the core ideas of assessment are best applied in unique, specific environments. 

It’s really taken away the capacity to engage meaningfully and as a result, our understanding of the word “National” has changed.  It used to mean inclusion and engagement in the work with anyone, anywhere, such that everyone could see the relevance of our work in their own unique context, to instead simply being about check-in points through surveys that provide a temperature reading of the assessment world. Much like others in the field of higher education, at a time of great community need, the community itself has become more distant and detached.   

How does that manifest in practical terms?  What exactly gets lost in the more granular parts of the work?

Assessment is very practitioner driven and, as such, we are often asked to implement or outline a simple, procedural, step-by-step guide for how to do it in any organization.  In the absence of authentic support that understands nuance, leaders look for simple, procedural assessment guidance rather than doing the deeper thinking and analysis that would make the discipline of assessment – as well as our organizations- better. 

The problem typically arises when an assessment director steps in and tries to implement someone else’s model – a model that was designed in a different context for a different set of people with a different way of working with different performance expectations designed for different students.  That’s when we end up fighting off our back foot.  Both we and the people being assessed realize immediately this is something brought in from outside and not designed with them or their unique design and needs in mind.  And that can breed resentment and a whole other level of push-back – something we are very familiar with when it comes to assessment.

My current work and my prior work with NILOA intentionally tries to avoid that approach or type of leadership.  We were very careful not to prescribe actions, but to focus instead more on understanding the landscape, unearthing the nuance, and developing reflective questions to help people think more deeply about their organizations and how they work before they move into implementing something. More “reflection before action” over “reflection in action,” if you will.

What were some of the other places assessment’s challenges were more visibly manifest in our schools and organizations during the pandemic?

The pandemic brought to light the number of places where assessment has not been at the table – either because they haven’t been invited or we didn’t feel it was our place to speak-up. So much was going on, so many inequities were elevated to the forefront, and so many of our staff were already limited in number and attention span and spread even more thin. That said, there were places where the voice of assessment was strikingly absent.

One great – or rather horrific – example is the way colleges went about testing and proctoring during the pandemic, specifically in the attempts to avoid cheating on exams by forcing students to download certain software, putting in restrictive time limitations, and attempting to implement highly controlled testing environments in virtual learning.  We all know the challenges with high stakes assessment in any environment, and faculty especially have learned very quickly over the last few years that using rigid, singular systems can’t be the leading indicator for anything – especially teacher evaluations.  But when our backs were against a wall, we had this complete knee-jerk, almost primitive response to creating environments of hyper-control to sustain the “integrity” and “rigor” of our curriculum that it just blew a lot of our credibility in the equity conversation.  It often worked at cross-purposes to the dialogue we have been so intentionally guiding about student engagement, seeing students as whole people, and embedding equity into assessment

I’ve tried to situate myself as a student in that type of environment numerous times and it’s very uncomfortable. When I think about it in the context of the other social unrest of this time it becomes almost unbearable. 

Yes – The testing question drives us right back to the issue of trauma and students who have been emotionally harmed by their schooling experience.  Colleges have worked very hard to undo years of damage from a system that was largely used as a blunt object or sorting mechanism rather than one by which to grow young people.  Add to this a period of heightened public police brutality, protests and growing inequality that directly aligns with race and you have essentially reverted the institution back into being a force for oppression rather than liberation. 

And that’s just one example of where assessment fell short in asserting its voice in the conversation about education and democracy writ large. We did well at the beginning about offering some basic frameworks for online learning, but as pandemic learning evolved, the student experience remained pretty terrible as we regressed back to old habits, unchecked assumptions about students as learners, and our roles as assessors.

How should we have responded and how can we guide conversations like these moving forward?

Places like NILOA and other national organizations perform a sort of double-duty.  On the one hand, they provide great tools and resources for leading these types of discussions.  But on a larger scale, they also provide political cover to people who are trying to push a point of view that is not being represented in the organization.  We let them say, “Look, this isn’t just me coming in here trying to ruffle feathers, but this is coming from national leadership who is having these conversations and debates across the country and it’s my duty to share with you that we are lacking a critical perspective.”

My view though is that if you are in a leadership position, it is your duty to step up and speak and ensure your leadership has these perspectives from the assessment community.  And whatever blow-back emerges needs to be managed as part of your leadership savvy – or better yet, managed on the front end so leadership sees that you offer a valuable point of view in an ongoing capacity rather than just a voice looking to be heard temporarily – more a permanent seat at the table as trusted counsel and advisor voice.

But just like the dangerous effort to lift a “one size fits all” procedure out of one organization and drop it into another, every organization has different politics too – especially around who gets heard, what leadership is receptive to, if their views are changeable, and when it might actually be advantageous over the long term to say less.  This is something that especially rings true in the pandemic – which is real for everyone.  Politics can be scary, especially when people are worried about their job security, healthcare, providing for their families, and paying the mortgage.  I understand why saying nothing may be a preferred course of action. The questions we raise are hard issues to begin with, and many leaders still view them as an inconvenience rather than the challenges they are hired and paid to solve. Issues about learning, equity, and student rights are at the crux of education, but we are hard pressed to see that clearly in times of crisis.   

It’s troubling to say it sounds as though some of the progress we’ve made as an assessment community may have come undone over the last year.  How do we rally back from this moment?

We need everyone involved and engaged and we have to start thinking more broadly about the larger implications of our work.  Assessment plays a critical role in delivering equitable learning and ensuring these institutions, which form the bedrock of American democracy, remain in place.  We have to be the people making sure they hold true to these ideals and not simply limit access or revert to the sorting mechanisms of old because leaders don’t know what to do, don’t recognize the dangers in not addressing it, or simply don’t see the problem.  This is difficult work, and we need to ensure our leaders are both aware of the ways in which reactionary behaviors manifest in schools and how to engage appropriately to defend against them.  We need people to join institutes like NILOA, the Assessment Institute at IUPUI, the ASSESS listserv community, and the others across the country because they are the best resources we have to develop communities and ensure we stay involved in these conversations. 

We also need organizations like CUNY to keep pushing – like this publication – to make these issues known and to have ongoing dialogue.  I hope people will respond to this interview – either in emails or other writing or commentaries in the Assessment Review and we can continually build our understanding of how best to move forward. 

Thank you so much for taking the time for this conversation.  We will certainly let you know how the Assessment Review community responds and hope you can come back to talk about it with us.

I’d be happy to.  Thank you again for the invitation!

Author(s)

Dr. Michael Seelig is an editor of The Assessment Review and Institutional Effectiveness Officer at Medgar Evers College, CUNY.