We Tell Ourselves Stories

Earlier this week, an essay written by Adam Kotsko was published in InsideHigherEd called “Making the Best of Assessment.” Two key sections of his piece stood out to me:

Would I really object if someone suggested that my institution might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if they are not working? I doubt that I would: in a certain sense it’s what every institution should be doing. Doing so systematically does bear significant costs in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in the form of data collection seems like a reasonable hedge against wasting time on efforts or approaches that don’t contribute to our mission. By the same token, getting into the habit of explaining why we’re doing what we’re doing can help us to avoid making decisions based on institutional inertia.

And a bit later on in the piece, referring to his own institution, department, and colleagues:
Despite that overall optimism, however, I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently. We all have limited energy and time, and so anything that can help us make sure we’re devoting our energy to things that are actually beneficial seems all to the good.

Recently, I experienced an assessment phenomenon that I’ve started to affectionately call “when data challenges our belief system.” (My colleague has also written about this here; when I was telling another colleague about this, she referred to it as “when reality interferes with our denial.”) I’ve been helping my colleagues work with assessment methods and their findings for many years now, but it only became apparent to me recently that sometimes we don’t want to believe what we see in our assessment findings or in other sources of data. Sometimes we’d prefer to just erase the evidence or the findings of a inquiry project rather than face the reality that we might be able to do something better, something different.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Joan Didion wrote a book with the title We tell ourselves stories in order to live.

In trying to build an evidence-informed culture for improvement in higher education, I have come to believe that our willingness to interrogate the stories we tell ourselves might be one of the biggest challenges we face. Resources? Yep: we certainly need those! Buy-in that the process is worthwhile? Totally important! Support and learning to enact effective assessment practices? Absolutely necessary. But …

We tell ourselves stories.

And guess what! I am totally guilty of this! Here’s a recent example: Using findings from a database-informed report I received almost 7 years ago, I believed that 95% of students who took an introductory course offered in my department went on to take many more courses at the university. This became my department’s story (because it was true — in 2006). It was a great story, until it wasn’t. When I requested and received an updated report, this is what we found out, and what we did:

Of 388 students from Summer 2009 – Fall 2013, 90 took only this course.  Thus, 23% of students who took the course didn’t take anything else. And although this means 77% of students did take other courses (we can celebrate that – it could have been worse, after all), we needed to think about whether or not our original story (remember it? 95%?) still held water. My departmental colleagues and I discussed this report, and once we came to grips with the new story the data was telling us, we realized that we wanted to make some advising process improvements with the goal of increasing the number of students who go on to take more courses at the university.

What’s challenging from an assessment perspective is that this practice of storytelling can totally limit our ability to use information to make improvements; more significantly, it can limit our learning.  Of course we should be analytical about our findings; we should understand their limitations, reliability and validity, the circumstances, the context. But if we believe everything to be perfectly fine — or even quite good — reliable findings that tell a different story can be hard to stomach. Sometimes what happens is folks blame the data; worse is that they blame the messenger; far, far worse is that they blame students.

We tell ourselves stories.

One of the greatest powers of assessment and of an evidence-informed, improvement-oriented culture is that it can foster critical reflection on practice, but only if we can be — if we’re willing to be — critically reflective. I also think assessment can foster really important conversations about students’ experiences and learning among colleagues (such as the great conversation we had in our department when the new data no longer supported our outdated story).  In other words: assessment itself can foster our learning.  But when we get stuck with our stories and we can’t see that there might be a different reality that’s out there, we shut learning out, and we shut out the opportunities that can result from learning.

No doubt that stories are important — to culture, to humanity, to life. As Brene Brown said in this TED talk, “stories are data with soul.”  I do think our stories can inform us and teach us (hey – I was an English major; of course I think this!) But I also think that if we’re not open to alternative views, to other voices (namely those of students), and to new information, then we’re being irresponsible educators because we’re not walking our talk. And so I’ve come to realize that a big part of my work is to support people in being willing to engage in critical reflection on practice, and to consider changing their stories if needed using the information they learn from assessment (it may come as no surprise to some of you that my work with Prior Learning Assessment and interest in heutagogy greatly inform my perspective and my approach). After all, education is supposed to be about learning; let’s use information we have to learn, and to get better.

So Many Posts; So Little … Posting

I have about 50 posts started, but none finished — or at least none worthy to post. As usual, I’ve been prattling (though only in my head, so far) about change in higher education; about the importance of leaders-as-learners and teachers; about technology, learning, assessment, teaching; about heutagogy; about access; etc. etc. etc.

So many posts; so little posting.

While not posting, I have been contemplating the lessons in two poems by Marge Piercy: The Seven of Pentacles and To Be of Use.

And I did take this picture of a lovely spring flower while out on a bike ride with my kid:

Meditation on a spring flower in the forest.

Meditation on a spring flower in the forest.

So there’s that.

Instead of posting my own ramblings right now, I’ll share a few links to others’ pieces that have really inspired my nog as of late.  I’ll also share a few quotes from each that I mentally highlighted – may they inspire you to inquire and reflect, too.

How Disruptive Is Information Technology Really? by Judith Ramaley – EDUCAUSE Review

The act of teaching is becoming more about designing the educational context and engaging students as they learn to approach material in more insightful and demanding ways. We are not transmitters of knowledge very often today, although an occasional superb lecture by a remarkably perceptive and even prescient speaker or a carefully crafted blog contribution can open up new ways of thinking about things.

We all know these simple things about how the educational experience is changing, but how recently have we paused to think about how truly wonderful it is to be able to use our smartphones to answer a question right immediately? My real concern is that not all questions have a quick, well-researched, and easy-to-find answer. Many, perhaps most, questions in today’s world are hard to formulate, are seen in very different ways by different people, or simply do not have good answers at all. That is why we still need real people who interact with each other in real time in order to frame questions that matter, to explore the ideas that come from those questions, and to work together to find solutions. No longer, however, are those people confined to the knowledge and experience that they carry in their own minds or that they can bring along with them on paper. They can tap into a true universe of material whenever they wish.

Good Teachers Become Less Important by Bernard Bull – Etale – Life in the Digital World.

When I think about the role of what we traditionally think of as teacher, one of the most important roles of teachers is to work hard at making themselves as unimportant as possible, not unimportant in the sense of lacking value, but unimportant in the sense that they are eventually no longer needed.  In other words, the goal of the teacher is to aid the learners in becoming self-directed learners.

Notes on social learning in business by Harold Jarche

Training and instruction are all about control, with curricula, sanctioned learning objectives, and performance criteria. This works when the field of study is knowable. But fewer fields remain completely knowable, if they ever were. Many institutions and professions have been built on the premise that knowledge can be transferred in some kind of controlled process. If you question that premise, you threaten people’s jobs, status, and sense of worth. This is why you see some violent reactions to the notion of informal and social learning having validity within organizations.

A major difference between communities of practice and work teams is that the former are voluntary. People want to join communities of practice. People feel affinity for their communities of practice. You know you are in a community of practice when it changes your practice.

And finally, a post from the Heutagogy Community of Practice: The PAH Continuum: Pedagogy, Andragogy, & Heutagogy, by Fred Garnett

When we collaboratively developed the ideas of the open context model of learning, Wilma Clark had pointed out that in Russia the word ‘obuchenie’ means both teaching and learning, and the PAH Continuum might be seen as a way of scaffolding ‘obuchenie’ as a move from teacher’s control to learner’s control. I would see it as axiomatic, as I did when I was ‘brokering’ learning, that teachers, whilst delivering their subject expertise, should be enabling learners to better understand the process of learning for themselves.

So many posts; so much learning. With great appreciation for all my teachers out there. As Bernard Bull wrote:

In one sense, a teacher is anyone or anything that contributes to our learning.

Moving Past Wishes

Many many many moons ago, when I was teaching English in my first “real” job in a community college in Washington state, I decided to try a technique called “negotiated assessment” that I’d heard about at a conference. The conference was about learning outcomes assessment — a fairly new paradigm at the time — and when I heard this idea I immediately wanted to try it. Why should I be the only one who assessed my students’ learning; why couldn’t they do it? Wouldn’t that help them pay more attention to their learning process and outcomes; wouldn’t it also, perhaps, potentially, lighten the power I wielded with my pen and my gradebook?

Grade A

Many thanks to Steve Snodgrass on Flickr for making this image available to use.

At the time it was a risky experiment for me: I didn’t yet have tenure, and here I was, giving the power of grading over to my students (except that I wasn’t — more about that below). My department chair (who was also the chair of my tenure committee) was supportive of the experiment, and so I tried it out. This was long before I knew much about learning or assessment; long before I knew how to support students in taking more ownership over their learning; long before I formally knew much about reflection, about deep learning, about metacognition, about self-directed and self-determined learning; long before I had devoured Freire and hooks and Dewey and Kegan and Daloz and Palmer and Doll and so many others; and it was long before I had enough experience  to really design the approach so it worked well.

Nonetheless, here’s how it went in my English 101 – College Composition course that spring semester:

  • I created the course description, the learning outcomes (some of which were departmentally prescribed), and the learning activities and assignments for the course.
  • I led and facilitated the course activities and provided direct instruction when needed.
  • Students engaged (or not, in some cases) in the writing processes I set up (and required), including peer review, responding to feedback on multiple drafts from me and other students, learning about revision as “re-seeing” and editing as cleaning up fly specks on the window so readers could see the message clearly, etc.
  • I never graded a single paper; I provided LOTS of feedback on LOTS of drafts, but I never assigned an actual letter grade.
  • At the end of the term, students wrote a final essay self-assessing their learning through the lens of the learning outcomes of the course and feedback they received from me and others along the way. They put the paper in the front of a binder that contained all of their other drafts and final papers (a very fundamental form of portfolio at the time). Based on their “analysis” and the “evidence of learning” in the binder, they recommended a final grade for themselves.
  • The last week of the term, I met with each student individually to review the paper, the portfolio, and to discuss the grade they thought they should get, and to talk about how they should transfer their learning from this class to other classes and contexts (because surely, if I told them to, they would, right?)

It was a fascinating experiment in all sorts of things — students whom I considered to be “strong” graded themselves much harder than I would have; students I considered to be “weak” (or “flakey”) recommended A grades for themselves. (I am now ashamed that I had those thoughts about my students at that time, but I did.) During the meeting, we “negotiated” the student’s grade and came to agreement, using the portfolio as an anchor, on what that grade should be. (In actual fact I still actually did hold most of the power. For one thing, I totally over-ruled 2 of my 20 students. They most certainly did not earn an A let alone demonstrate much learning. What kind of pushover did they think I was!?!)

I was reminded of this experiment from many moons ago this past week when I read a new perspectives post about heutagogy contributed by Fred Garnett in the Heutagogy Community of Practice.  Read it here:

The PAH Continuum: Pedagogy, Andragogy & Heutagogy

In this post, Fred describes his practice of  “brokering:”

Writing the syllabus and developing the schedule of delivery along with the work to be completed meant that I was, in effect, building the framework of what I was teaching. Consequently I really understood what the boundaries were and so could better broker between the formal requirements of the education system and the personal desires of my learners; I had found that all these ‘failing’ students wanted to learn.  On the social impact course each student picked any technology that interested them to research and write about. I showed them how to “play” with the learning requirements, which can be used as creative constraints, and how best to meet them in their completed work. I also encouraged them to present that work in original ways rather than as just a written report. Although most presented reports a precious few tried original approaches, such as wall charts, cartoons, a class presentation with Q&A, and so on. Most importantly simply having the opportunity to present finished written work in ways that they determined meant that they thought about various ways in which to explain their ideas.

Here’s what I like about this concept and what I wish I had done in that class: I wish I had actually tapped into and understood the personal desires of my learners and showed them how to “play” with the learning requirements and how to best meet them in their work. I wish that I had encouraged them to produce artifacts that were original — that were personalized to their desires — that would meet the learning outcomes. And I wish that I had understood that my “failing” students — those that I unfairly deemed flakey, or lazy, or poor writers — probably did want to learn, but maybe not the way I thought they should. I wish … I wish … I wish…

This is why I am excited about heutagogy and wanted to start a Community of Practice focusing on it — it so nicely grounds and frames the approaches I try to use when I teach in a way that does not leave me wishing. It provides a way for me to organize and re-vision and re-enact my knowledge about self-directed learning; about authentic assessment; about learning outcomes, course design, and curricular structure; and about my preferred “teaching” approach.

I am so grateful to continue to learn about my practice — and to self-determine my path. Thanks to all who’ve joined the Heutagogy Community of Practice for already furthering my learning in ways I hadn’t ever anticipated.


To Africa

Lake Nakuru National Park, Kenya

Lake Nakuru National Park, Kenya

In October, I had the amazing opportunity to travel to Kenya to facilitate a week-long faculty and staff workshop about Prior Learning Assessment at Tangaza College in Nairobi.  I am grateful to my colleagues at the School for New Learning of DePaul University for inviting me to participate!

In a bookstore in Nairobi, I purchased Beryl Markham’s autobiographical account of her time in Africa titled West with the Night.  Here’s a particularly relevant and poignant paragraph that captured my attention:

“There are as many Africas as there are books about Africa — and as many books about it as you could read in a leisurely lifetime. Whoever writes a new one can afford a certain complacency in the knowledge that his is a new picture agreeing with no one else’s, but likely to be haughtily disagreed with by all those who believed in some other Africa. … Being thus all things to all authors, it follows, I suppose, that Africa must be all things to all readers. . . . Africa is mystic; it is wild; it is a sweltering inferno; it is a photographer’s paradise, a hunter’s Valhalla, an escapist’s Utopia. It is what you will, and it withstands all interpretations. It is the last vestige of a dead world or the cradle of a shiny new one. ”
― Beryl Markham, West with the Night

Read more about my trip here: Out to Africa


Teaching Deep

Just a quick share from the IDEA center, this great little white paper called Promoting Deep Learning, by Barbara J. Millis (the Director of the Teaching and Learning Center at The University of Texas at San Antonio). Here’s a snippit to pique your interest:

To result in deep learning, however, faculty must carefully sequence activities — either in class or online — to provide the student active learning and interactions as identified in the deep-learning model. Students must DO something with the work prepared outside of class. Designing sequences that enable students to approach the same material in multiple ways also builds on the science of human learning cited above. Homework is thus not an artificial assignment stuffed into a teacher’s briefcase for later grading. It becomes the foundation for a meaningful sequence to further deep learning.

There are great examples of graphic organizers in this article too. I’m a big fan of such tools for helping learners make sense of what they’re grappling with — we use them all the time in our PLA program and I use them in faculty development workshops I facilitate as well. In fact, I am using this one in a faculty development workshop about PLA that I am facilitating next month at Tangaza College in Nairobi, Kenya:

Experience to Learning Chronolog

The chronolog is a learning tool we use in our PLA program to help students “discover” (alone, with others outside of class, and with each other in class) what they already know and can do and to make sense of how their life and work experiences contributed to their learning areas. I’m playing with the final column in this version — asking about the significance of their learning to their lives now — and depending on how that works out, we might incorporate this into the chronolog we use with our PLA students.

This book — Learning and Awareness — is about students’ different approaches to learning (surface, strategic, and deep), and was a key text in helping me frame my own dissertation research into developing self-directed learning (and a key text in helping me think about teaching deep). There are intersections among these deep approaches and heutagogical learning theory, for what  deep learning comes down to is actually heutagogical in nature: we’re helping students develop their capability as learners. Teaching deep is not about helping students acquire content knowledge or demonstrate competency (in skills and knowledge) alone. These aspects of learning are important, no doubt, but temporary and fleeting if not taught deep. Teaching deep is about setting students up to be effective, capable learners.  As Blaschke (2012, p. 59) summarizes, capable learners demonstrate:

• self-efficacy, in knowing how to learn and continuously reflect on the learning process;
• communication and teamwork skills, working well with others and being openly communicative;
• creativity, particularly in applying competencies to new and unfamiliar situations and by being adaptable and flexible in approach;
• positive values (Hase & Kenyon, 2000; Kenyon & Hase, 2010; Gardner et al., 2007).

Teaching deep can help develop these capabilities; and these capabilities are assets far more valuable than any job, house, car, or retirement package will ever be.


Blaschke, L.M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. International Review of Research in Open and Distance Learning, 13(1), 56-71. Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/1076/2113


Fearing Assessment; Fearing Learning (And Fearing David Brooks)

O this learning, what a thing it is! ~Grumio in Taming of the Shrew, William Shakespeare

Twice this past month I’ve heard the word “fear” used by faculty when referring to their experience of assessing student learning in their courses. One person described it as fear of students disagreeing with their grade or feedback, or generally unhappy with the judgment the instructor made about their work and requesting explanation and justification (much of which could be alleviated, I thought, if the instructor made the criteria transparent to students, or even better, if the criteria were collectively developed with students, but I digress…).

Thanks to Jimee, Jackie, Tom & Asha on Flickr for making this photo available for use.

The other person described her fear that she lacks the ability to discern quality and to really be able to tell what a student has learned. She described her lack of confidence in using a writing rubric to “judge” what about a student’s writing, as exemplified in a single assignment, is exceptional and what is developing (and every shade of grey in between). I appreciated her honesty with this challenge; I’ve certainly faced it as well (though in my case, “fear” was not a word I used to describe what I experienced as a “bleepin’ assessment conundrum!”). Nonetheless, her description reminded me of something from Parker Palmer’s The Courage to Teach: “Teaching is a daily exercise in vulnerability.”

So, too, is learning.

I am pretty certain that learners fear assessment as well, which is truly unfortunate and totally not necessary, and in the end, adversely affects our ability to learn. When faculty work from a “gotcha” perspective, of course assessment is something to fear! I remember a Shakespeare course I took in college that made me have night terrors; I couldn’t sleep that term because that class and that prof were seriously scary. Our final grade consisted of our scores on 5 tests: a test after each unit (comedy, drama, history, and what was the other??? – poetry, I guess), and the big ugly final exam, 3 hours of closed book / closed notes mental torture. These tests were tricky because they were designed as “gotcha” tests (including the essay part of the tests, for which we could use only one side of a single piece of unlined white 8.5 x 11″ paper to address the topic, for no clear reason other than the prof didn’t want to read more than what could fit in this designated space). It was always obvious from the smirk on his face and comments under his moldy breath that the curmudgeonly old prof enjoyed this process. These tests didn’t in any way advance or enhance my learning (I memorized a lot of Shakespeare that term but I didn’t learn any of it except for a few random quotes I can pull out of my head for cocktail parties or blog posts); they didn’t help me appreciate Shakespeare in new ways, or connect important themes or ideas to topics I was interested in. They freaked me out! Why was that necessary?

And now there is something else to fear: David Brooks, writing this op-ed piece in The New York Times, has called for value-added assessments.

Colleges have to test more to find out how they’re doing . . . There has to be some way to reward schools that actually do provide learning and punish schools that don’t. . . This is the beginning of college reform.

To which I reply: ARE YOU SERIOUS? How will THIS advance learning on the part of students, and on the part of faculty and institutions? Punish schools that don’t. Really? Punishment creates fear; punishment creates distrust. And fear and distrust do not promote learning — for students or any of us! I don’t disagree that we need to know how we’re doing … we do! We really, really do! But I absolutely believe that this approach is completely antithetical to actually promoting learning (note that I didn’t say “producing” learning, the term Brooks used, as if learning were something that gets assembled on a conveyor belt). This approach will foster fear; fear inhibits learning. Period.

Colleges (and faculty) have to remove fear first — this should be the beginning of college reform. I think it was Shakespeare who once wrote:

Things done well and with a care, exempt themselves from fear.


This Makes Us Official

Marylhurst University

My colleagues in the Assessment Program and I finally got our program description and content on the Marylhurst website. We’ve been wanting to do so for a very long time, but we needed the time and space to get it all organized. Thanks to the great work of our Educational Assessment Specialist Sione, we’re now present, virtually.

I think that makes us official now, right?

Check it out:  Assessment at Marylhurst University