Fearing Assessment; Fearing Learning (And Fearing David Brooks)

O this learning, what a thing it is! ~Grumio in Taming of the Shrew, William Shakespeare

Twice this past month I’ve heard the word “fear” used by faculty when referring to their experience of assessing student learning in their courses. One person described it as fear of students disagreeing with their grade or feedback, or generally unhappy with the judgment the instructor made about their work and requesting explanation and justification (much of which could be alleviated, I thought, if the instructor made the criteria transparent to students, or even better, if the criteria were collectively developed with students, but I digress…).

Thanks to Jimee, Jackie, Tom & Asha on Flickr for making this photo available for use.

The other person described her fear that she lacks the ability to discern quality and to really be able to tell what a student has learned. She described her lack of confidence in using a writing rubric to “judge” what about a student’s writing, as exemplified in a single assignment, is exceptional and what is developing (and every shade of grey in between). I appreciated her honesty with this challenge; I’ve certainly faced it as well (though in my case, “fear” was not a word I used to describe what I experienced as a “bleepin’ assessment conundrum!”). Nonetheless, her description reminded me of something from Parker Palmer’s The Courage to Teach: “Teaching is a daily exercise in vulnerability.”

So, too, is learning.

I am pretty certain that learners fear assessment as well, which is truly unfortunate and totally not necessary, and in the end, adversely affects our ability to learn. When faculty work from a “gotcha” perspective, of course assessment is something to fear! I remember a Shakespeare course I took in college that made me have night terrors; I couldn’t sleep that term because that class and that prof were seriously scary. Our final grade consisted of our scores on 5 tests: a test after each unit (comedy, drama, history, and what was the other??? – poetry, I guess), and the big ugly final exam, 3 hours of closed book / closed notes mental torture. These tests were tricky because they were designed as “gotcha” tests (including the essay part of the tests, for which we could use only one side of a single piece of unlined white 8.5 x 11″ paper to address the topic, for no clear reason other than the prof didn’t want to read more than what could fit in this designated space). It was always obvious from the smirk on his face and comments under his moldy breath that the curmudgeonly old prof enjoyed this process. These tests didn’t in any way advance or enhance my learning (I memorized a lot of Shakespeare that term but I didn’t learn any of it except for a few random quotes I can pull out of my head for cocktail parties or blog posts); they didn’t help me appreciate Shakespeare in new ways, or connect important themes or ideas to topics I was interested in. They freaked me out! Why was that necessary?

And now there is something else to fear: David Brooks, writing this op-ed piece in The New York Times, has called for value-added assessments.

Colleges have to test more to find out how they’re doing . . . There has to be some way to reward schools that actually do provide learning and punish schools that don’t. . . This is the beginning of college reform.

To which I reply: ARE YOU SERIOUS? How will THIS advance learning on the part of students, and on the part of faculty and institutions? Punish schools that don’t. Really? Punishment creates fear; punishment creates distrust. And fear and distrust do not promote learning — for students or any of us! I don’t disagree that we need to know how we’re doing … we do! We really, really do! But I absolutely believe that this approach is completely antithetical to actually promoting learning (note that I didn’t say “producing” learning, the term Brooks used, as if learning were something that gets assembled on a conveyor belt). This approach will foster fear; fear inhibits learning. Period.

Colleges (and faculty) have to remove fear first — this should be the beginning of college reform. I think it was Shakespeare who once wrote:

Things done well and with a care, exempt themselves from fear.


Connecting Paper And Image: Assessment As Origami

Thanks to Claudia M&M on Flickr for making this image available for use.

Joshua Brown, the editor of Research & Practice in Assessment (published by the Virginia Assessment Group), wrote in his From The Editor column in the Winter 2011 issue this interesting idea about assessment paradigms:

Whereas Western art focuses upon the freedom to move images around on paper or canvas to create fixed patterns, origami ignores the separation between the image and the paper. The paper becomes part of the image, and is twisted and folded until it is the picture, not merely the surface on which it lies. -John D. Barrow, The Artful Universe

Just as the artist of origami has a different approach to perceiving the relationship between image and paper, the thematic focus of this issue invites inquiry as to whether assessment might adopt similar connecting paradigms. In establishing and executing assessment initiatives, there are places where our focus is predominantly one of separation – our rubrics have multiple levels of competencies, item correlation allows us to maximize the efficiency of our scales, and purpose statements or objectives are arranged in a structured hierarchy. We strive for increased validity and reliability, but even good research techniques possess implications regarding their social, psychological, and educational contexts. There is an ongoing tension between focusing on the trees while at the same time giving appropriate attention to the forest.

As such, it is worth considering, to what extent can assessment also function as a mechanism that connects broader realms rather than one which at times is noted for solely focusing on measurement or standardization? In addition to its dominant descriptive or defining properties, is it possible for assessment to also possess generative properties? [bold added here for emphasis] I am not positing these philosophical assessment questions to establish rigid dichotomies. In fact, it may be more beneficial for me to ask these of my own assessment practices. While aiming to achieve the utilitarian ideals of efficiency and effectiveness, is it also possible for me to construct my assessments in a manner that advances good human behavioral, educational, and social theory? Is it really possible for me to look at a Scantron sheet in a manner that resembles the philosophical paradigm of the origami artist?

The paradigm of the origami artist … assessment as generative, as learning … paper and image as one … learningteachingassessmentlearningteachingassessmentlearning.

This poses assessment as a part of learning; learning as a part of assessment; the two entwined in meaningful ways. Not assessment of learning, but for and as learning.

From here on out I will see myself as an origami artist, connecting paper and image to become one, to generate, to advance, to learn.  In fact, perhaps the best use of a Scantron sheet might be to fold it into a bird so that it might fly away … far, far away.

Being Intentional About Being Intentional

Intentionality. With thought. Deliberate. Designed. Purposeful.

Intentionality is on my mind a lot because I think that assessment can be more interesting, engaging, and powerful (for learners and teachers) when it’s less about measurement and accountability and more about supporting authentic learning practices. In this vein, assessment can be an interesting catalyst for reminding us to be intentional in what we do and how we do it — and in knowing why we do what we do.  Being intentional means thinking about each and every aspect of a course we teach or program we facilitate to do our best to ensure it lines up to what we’re hoping people will learn from it. The short article How To Study Your Own Teaching (And Why You Might Want To), by Janine Utell, speaks to this very idea:

I’ve developed strategies to create good discussion, to facilitate broad and deep involvement, and to synthesize the contributions of the classroom community. I feel like my classes are going best when the room is a bit rowdy, when interactions lead to debates and eurekas. But due partly to assessment work on my campus and partly to collaboration with colleagues in a different discipline around designing a study of student writing, I decided to create a project of my own to investigate the effectiveness of my practice. I wanted a more robust picture of what’s going on in my classroom and whether it’s working.

Utell wants to see her practice differently because she wants to make sure what she does is working. (And like many of my best teachers and colleagues, reflecting on her teaching practice and pedagogical commitments is likely part of her DNA and happens with less intentionality as well).

Intentionality. With thought. Deliberate. Designed. Purposeful.

Intentionality certainly has its role in my yoga practice as well. In early January, my yoga teacher asked a group of us to define one thing we each wanted to focus on this coming year in our practice.  It was a sort of New Year’s Resolution moment. And I knew right away:  I need to focus on squaring my hips in poses such as Warrior 3 or Pyramid.

This is not me - my hips are not this square. Nor do I practice yoga in a place like this. I live in Portland - we practice rain yoga. So thanks to rfarmer on Flickr for making this image available for use!

My hips always want to go way off to the side, and I thus don’t get the benefit of the pose when that happens. By stating this intention, and with self-assessment and my teacher’s coaching and assessment in each session, I maintain that intentionality, and I am improving. I can feel it. It’s on my mind constantly in any pose that requires me to get squared. If yoga is about anything for me, it’s about intentionality. And it’s about seeing myself differently.

Intentionality. With thought. Deliberate. Designed. Purposeful.

Parenting my 4-year old is also a practice in intentionality — and learning and assessment — as well. Let me illustrate:

A few weeks ago Mac and I went to the zoo and found a friend in this fellow. Mac had been having a hard time with his own swimming attempts recently, freaking out at the thought of going under the water, so I seized the moment:

Me: Mac, why don’t you ask him why he likes to swim underwater so much.

Mac: Mr. Sea Lion, why do you like to swim underwater so much?

Mr. Sea Lion: blurb blurp bubble blurp

Mac: Mama, he says it’s fun to swim to the bottom and see all the kids down here.

Me: Wow, neat! I wonder what you can see when you swim to the bottom of the pool.

Mac: That’s silly mama. I can’t see anything. My eyes are closed!

WHAT? Mac always wore goggles in the pool – why did he close his eyes? What did he think the goggles were for? A fashion statement? To hold his hair back? I pointed out that with his goggles on he could open his eyes and see the bottom just like the sea lion, and – EPIPHANY!

Mac: Really? If I open my eyes in my goggles I can see down there?

Me: Yep – and you won’t get water in your eyes!

Mac: Cooooool! That will be really good, mama!

And the next time in the water, with his goggles on, he opened his eyes, swam to the bottom, and fetched a toy. Just like Mr. Sea Lion. Intentionality helped us out here, again. It reminded me that we often take things for granted and don’t question them for a long time until we have opportunity to see them differently.

Intentionality. With thought. Deliberate. Designed. Purposeful.

It’s good to see things differently – with our eyes open.  (If you’re in need, Mac has a pair of goggles he’ll let you borrow if you’d like – if you open your eyes under there, you might be amazed at what you’ll find.)

Prattle Your Nog For A Good Cause

Friends of PrattleNog –

I know you all want to get your nogs prattling – here’s a good opportunity for a great cause!

Close enough...Thanks for J from the UK on Flickr for making this image available.

My Marylhurst colleague Jesse Stommel is seeking feedback on his ideas for our new online English degree program. As he writes in Hybrid Pedagogy,

I’ve been thinking about my audience for this series of posts. Initially, I had thought to bring digital humanities, literary studies, and educational technology experts into conversation, allowing my ideas for the program to be considered and influenced by a much larger network. I’m realizing, though, that there’s another group of experts from whom I particularly want feedback and suggestions: students. Ideally, this would include input from prospective students for the program, but since the program is only just barely beginning to germinate, what I’d like to do here is askboth students and teachers in existing programs to think about how the English degree is being transformed by digital technologies and about how online learning can be re-imagined through the use of new (and increasingly social) media.

I am super excited about this program in part because I will aways be an English major in heart and by degree (ok, by two degrees), and I love Jesse’s approach and ideas for getting this program off the ground. (We’re also going to do some nogging about innovative and meaningful ways to assess this program and students’ learning – good times to come!)

So – go HERE and help him out! It’s for a good cause, and I guarantee it will get your nog prattling!

Hairballs (Yes, This Is About Teaching, Learning, And Assessment)

The other night we hosted a grown-up dinner party (ok, ok, it was a potluck) for the parents of my kid’s classmates. I thought it would be nice to get to know these folks given that our kids might be together for the next several years and that between the hustle and bustle of pick-up and drop-off we know each other’s cars, but not each other’s names, let alone which parents belong to which kids.

In the midst of the getting-to-know you conversations, I shared that I was the Dean of Learning & Assessment at Marylhurst University, and this is what I got from one parent:


And she made this awful choking sound, like she had a hairball stuck in her throat.

Here is a picture of my cats instead of a picture of their hairballs. Because that's the kind of blogger I am.

Huh? How can you “hate” assessment? Hating eggplant? I get that! Not really enjoying a boring baseball game? TOTALLY!

But assessment? Huh?

I really didn’t expect or understand her reaction! I mean, for me, good assessment is an act of care. It’s also an act of learning! It’s certainly not the bane of my existence. (Understatement!)

But in all fairness, that’s what I call “good” assessment — when assessment furthers the learning of our students, ourselves, and our institutions. There is also “bad” assessment … and this is likely what she has experienced. And I can totally understand that like a bad digestive system or a bad meal, one might have a hairball-like reaction to bad assessment.

I had a conversation with a colleague last week in which he differentiated good and bad assessment using the words “weak” and “strong.” Weak assessment doesn’t really tell us anything meaningful, and though it might be easier to implement (e.g., a standardized test that spits results into a spreadsheet), it may not help learners learn or teachers teach. Strong assessment, on the other hand, is useful; it helps learners, it helps instructors, it helps programs, and it helps institutions.

I’m not a fan of the dichotomies of bad / good, weak / strong, hate / like . And I thought that there might be an analogy in how people view and do assessment from learning theory, namely, that people have different conceptions of learning (thanks to my dissertation friends Marton and Booth, no relation, I get this!). If people have different conceptions of learning, then of course they would have different conceptions of assessment!

So let’s try this on. Check out a summary of five different conceptions of learning from one study, here:

Conceptions of Learning

Now – let’s take this framework one further step and apply it to assessment.

1. If learning is seen as an increase in knowledge, then assessment would attempt to measure that increase. Thus, we might do a pre-test, a post-test, and look at the difference. Ho hum…

2. If learning is seen as memorizing, then assessment would attempt to measure what students have memorized. Thus, an assessment would likely ask students to “spit back” what they “learned.” (Speaking of hairballs…)

3. If learning is seen as acquiring facts, skills, and methods that can be applied later, then assessment might be more authentic, seeing if students can actually apply their learning to “real-world” projects or problems. So we might ask students to write an essay to see if they can write, instead of take a multiple-choice test to see if they can write. (Right?) Now we are getting somewhere, though still more weak than strong on the continuum, perhaps.

4. If learning is seen as deep process for meaning-making, whereby learners relate content to themselves and their worlds, then assessment would likely support this learning process. Again, authentic assessment opportunities — like writing a persuasive letter to their congressperson to demonstrate their ability to construct a well-informed written argument — come to mind. We also might delve into self-assessment here as well. (Let the angels sing!)

5. If learning is seen as transformative (btw: AWESOME!), that is “interpreting and understanding reality in a different way,” then assessment would attempt to capture this re-interpretation and transformation. And man – that’s sure hard to do! And it usually must involve and value self-assessment. YAY! And wow – how meaningful this can be to both learner and instructor!

This might be one explanation why different individuals — as well as different disciplines and different institutions  — see assessment differently. (Might…or might not. Afterall, this blog is called PrattleNog for a reason, people! I’m just thinkin’ outloud here.)

Or perhaps I have gone too far. Maybe people just hear that word and automatically get hair stuck in their throat. Which is too bad, because coughing up (or cleaning up!) a hairball is rarely a learning experience. And assessment — when done strong*, right*, and good* — is.

(*Note: Grammar is intentionally messed up. I’m an English teacher at heart and by degree; I know it should be “strongly,” “rightly,” and “goodly.”)

This Makes Us Official

Marylhurst University

My colleagues in the Assessment Program and I finally got our program description and content on the Marylhurst website. We’ve been wanting to do so for a very long time, but we needed the time and space to get it all organized. Thanks to the great work of our Educational Assessment Specialist Sione, we’re now present, virtually.

I think that makes us official now, right?

Check it out:  Assessment at Marylhurst University