We Tell Ourselves Stories

Earlier this week, an essay written by Adam Kotsko was published in InsideHigherEd called “Making the Best of Assessment.” Two key sections of his piece stood out to me:

Would I really object if someone suggested that my institution might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if they are not working? I doubt that I would: in a certain sense it’s what every institution should be doing. Doing so systematically does bear significant costs in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in the form of data collection seems like a reasonable hedge against wasting time on efforts or approaches that don’t contribute to our mission. By the same token, getting into the habit of explaining why we’re doing what we’re doing can help us to avoid making decisions based on institutional inertia.

And a bit later on in the piece, referring to his own institution, department, and colleagues:
Despite that overall optimism, however, I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently. We all have limited energy and time, and so anything that can help us make sure we’re devoting our energy to things that are actually beneficial seems all to the good.

Recently, I experienced an assessment phenomenon that I’ve started to affectionately call “when data challenges our belief system.” (My colleague has also written about this here; when I was telling another colleague about this, she referred to it as “when reality interferes with our denial.”) I’ve been helping my colleagues work with assessment methods and their findings for many years now, but it only became apparent to me recently that sometimes we don’t want to believe what we see in our assessment findings or in other sources of data. Sometimes we’d prefer to just erase the evidence or the findings of a inquiry project rather than face the reality that we might be able to do something better, something different.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Joan Didion wrote a book with the title We tell ourselves stories in order to live.

In trying to build an evidence-informed culture for improvement in higher education, I have come to believe that our willingness to interrogate the stories we tell ourselves might be one of the biggest challenges we face. Resources? Yep: we certainly need those! Buy-in that the process is worthwhile? Totally important! Support and learning to enact effective assessment practices? Absolutely necessary. But …

We tell ourselves stories.

And guess what! I am totally guilty of this! Here’s a recent example: Using findings from a database-informed report I received almost 7 years ago, I believed that 95% of students who took an introductory course offered in my department went on to take many more courses at the university. This became my department’s story (because it was true — in 2006). It was a great story, until it wasn’t. When I requested and received an updated report, this is what we found out, and what we did:

Of 388 students from Summer 2009 – Fall 2013, 90 took only this course.  Thus, 23% of students who took the course didn’t take anything else. And although this means 77% of students did take other courses (we can celebrate that – it could have been worse, after all), we needed to think about whether or not our original story (remember it? 95%?) still held water. My departmental colleagues and I discussed this report, and once we came to grips with the new story the data was telling us, we realized that we wanted to make some advising process improvements with the goal of increasing the number of students who go on to take more courses at the university.

What’s challenging from an assessment perspective is that this practice of storytelling can totally limit our ability to use information to make improvements; more significantly, it can limit our learning.  Of course we should be analytical about our findings; we should understand their limitations, reliability and validity, the circumstances, the context. But if we believe everything to be perfectly fine — or even quite good — reliable findings that tell a different story can be hard to stomach. Sometimes what happens is folks blame the data; worse is that they blame the messenger; far, far worse is that they blame students.

We tell ourselves stories.

One of the greatest powers of assessment and of an evidence-informed, improvement-oriented culture is that it can foster critical reflection on practice, but only if we can be — if we’re willing to be — critically reflective. I also think assessment can foster really important conversations about students’ experiences and learning among colleagues (such as the great conversation we had in our department when the new data no longer supported our outdated story).  In other words: assessment itself can foster our learning.  But when we get stuck with our stories and we can’t see that there might be a different reality that’s out there, we shut learning out, and we shut out the opportunities that can result from learning.

No doubt that stories are important — to culture, to humanity, to life. As Brene Brown said in this TED talk, “stories are data with soul.”  I do think our stories can inform us and teach us (hey – I was an English major; of course I think this!) But I also think that if we’re not open to alternative views, to other voices (namely those of students), and to new information, then we’re being irresponsible educators because we’re not walking our talk. And so I’ve come to realize that a big part of my work is to support people in being willing to engage in critical reflection on practice, and to consider changing their stories if needed using the information they learn from assessment (it may come as no surprise to some of you that my work with Prior Learning Assessment and interest in heutagogy greatly inform my perspective and my approach). After all, education is supposed to be about learning; let’s use information we have to learn, and to get better.
Advertisements

So Many Posts; So Little … Posting

I have about 50 posts started, but none finished — or at least none worthy to post. As usual, I’ve been prattling (though only in my head, so far) about change in higher education; about the importance of leaders-as-learners and teachers; about technology, learning, assessment, teaching; about heutagogy; about access; etc. etc. etc.

So many posts; so little posting.

While not posting, I have been contemplating the lessons in two poems by Marge Piercy: The Seven of Pentacles and To Be of Use.

And I did take this picture of a lovely spring flower while out on a bike ride with my kid:

Meditation on a spring flower in the forest.

Meditation on a spring flower in the forest.

So there’s that.

Instead of posting my own ramblings right now, I’ll share a few links to others’ pieces that have really inspired my nog as of late.  I’ll also share a few quotes from each that I mentally highlighted – may they inspire you to inquire and reflect, too.

How Disruptive Is Information Technology Really? by Judith Ramaley – EDUCAUSE Review

The act of teaching is becoming more about designing the educational context and engaging students as they learn to approach material in more insightful and demanding ways. We are not transmitters of knowledge very often today, although an occasional superb lecture by a remarkably perceptive and even prescient speaker or a carefully crafted blog contribution can open up new ways of thinking about things.

We all know these simple things about how the educational experience is changing, but how recently have we paused to think about how truly wonderful it is to be able to use our smartphones to answer a question right immediately? My real concern is that not all questions have a quick, well-researched, and easy-to-find answer. Many, perhaps most, questions in today’s world are hard to formulate, are seen in very different ways by different people, or simply do not have good answers at all. That is why we still need real people who interact with each other in real time in order to frame questions that matter, to explore the ideas that come from those questions, and to work together to find solutions. No longer, however, are those people confined to the knowledge and experience that they carry in their own minds or that they can bring along with them on paper. They can tap into a true universe of material whenever they wish.

Good Teachers Become Less Important by Bernard Bull – Etale – Life in the Digital World.

When I think about the role of what we traditionally think of as teacher, one of the most important roles of teachers is to work hard at making themselves as unimportant as possible, not unimportant in the sense of lacking value, but unimportant in the sense that they are eventually no longer needed.  In other words, the goal of the teacher is to aid the learners in becoming self-directed learners.

Notes on social learning in business by Harold Jarche

Training and instruction are all about control, with curricula, sanctioned learning objectives, and performance criteria. This works when the field of study is knowable. But fewer fields remain completely knowable, if they ever were. Many institutions and professions have been built on the premise that knowledge can be transferred in some kind of controlled process. If you question that premise, you threaten people’s jobs, status, and sense of worth. This is why you see some violent reactions to the notion of informal and social learning having validity within organizations.

A major difference between communities of practice and work teams is that the former are voluntary. People want to join communities of practice. People feel affinity for their communities of practice. You know you are in a community of practice when it changes your practice.

And finally, a post from the Heutagogy Community of Practice: The PAH Continuum: Pedagogy, Andragogy, & Heutagogy, by Fred Garnett

When we collaboratively developed the ideas of the open context model of learning, Wilma Clark had pointed out that in Russia the word ‘obuchenie’ means both teaching and learning, and the PAH Continuum might be seen as a way of scaffolding ‘obuchenie’ as a move from teacher’s control to learner’s control. I would see it as axiomatic, as I did when I was ‘brokering’ learning, that teachers, whilst delivering their subject expertise, should be enabling learners to better understand the process of learning for themselves.

So many posts; so much learning. With great appreciation for all my teachers out there. As Bernard Bull wrote:

In one sense, a teacher is anyone or anything that contributes to our learning.

Moving Past Wishes

Many many many moons ago, when I was teaching English in my first “real” job in a community college in Washington state, I decided to try a technique called “negotiated assessment” that I’d heard about at a conference. The conference was about learning outcomes assessment — a fairly new paradigm at the time — and when I heard this idea I immediately wanted to try it. Why should I be the only one who assessed my students’ learning; why couldn’t they do it? Wouldn’t that help them pay more attention to their learning process and outcomes; wouldn’t it also, perhaps, potentially, lighten the power I wielded with my pen and my gradebook?

Grade A

Many thanks to Steve Snodgrass on Flickr for making this image available to use.

At the time it was a risky experiment for me: I didn’t yet have tenure, and here I was, giving the power of grading over to my students (except that I wasn’t — more about that below). My department chair (who was also the chair of my tenure committee) was supportive of the experiment, and so I tried it out. This was long before I knew much about learning or assessment; long before I knew how to support students in taking more ownership over their learning; long before I formally knew much about reflection, about deep learning, about metacognition, about self-directed and self-determined learning; long before I had devoured Freire and hooks and Dewey and Kegan and Daloz and Palmer and Doll and so many others; and it was long before I had enough experience  to really design the approach so it worked well.

Nonetheless, here’s how it went in my English 101 – College Composition course that spring semester:

  • I created the course description, the learning outcomes (some of which were departmentally prescribed), and the learning activities and assignments for the course.
  • I led and facilitated the course activities and provided direct instruction when needed.
  • Students engaged (or not, in some cases) in the writing processes I set up (and required), including peer review, responding to feedback on multiple drafts from me and other students, learning about revision as “re-seeing” and editing as cleaning up fly specks on the window so readers could see the message clearly, etc.
  • I never graded a single paper; I provided LOTS of feedback on LOTS of drafts, but I never assigned an actual letter grade.
  • At the end of the term, students wrote a final essay self-assessing their learning through the lens of the learning outcomes of the course and feedback they received from me and others along the way. They put the paper in the front of a binder that contained all of their other drafts and final papers (a very fundamental form of portfolio at the time). Based on their “analysis” and the “evidence of learning” in the binder, they recommended a final grade for themselves.
  • The last week of the term, I met with each student individually to review the paper, the portfolio, and to discuss the grade they thought they should get, and to talk about how they should transfer their learning from this class to other classes and contexts (because surely, if I told them to, they would, right?)

It was a fascinating experiment in all sorts of things — students whom I considered to be “strong” graded themselves much harder than I would have; students I considered to be “weak” (or “flakey”) recommended A grades for themselves. (I am now ashamed that I had those thoughts about my students at that time, but I did.) During the meeting, we “negotiated” the student’s grade and came to agreement, using the portfolio as an anchor, on what that grade should be. (In actual fact I still actually did hold most of the power. For one thing, I totally over-ruled 2 of my 20 students. They most certainly did not earn an A let alone demonstrate much learning. What kind of pushover did they think I was!?!)

I was reminded of this experiment from many moons ago this past week when I read a new perspectives post about heutagogy contributed by Fred Garnett in the Heutagogy Community of Practice.  Read it here:

The PAH Continuum: Pedagogy, Andragogy & Heutagogy

In this post, Fred describes his practice of  “brokering:”

Writing the syllabus and developing the schedule of delivery along with the work to be completed meant that I was, in effect, building the framework of what I was teaching. Consequently I really understood what the boundaries were and so could better broker between the formal requirements of the education system and the personal desires of my learners; I had found that all these ‘failing’ students wanted to learn.  On the social impact course each student picked any technology that interested them to research and write about. I showed them how to “play” with the learning requirements, which can be used as creative constraints, and how best to meet them in their completed work. I also encouraged them to present that work in original ways rather than as just a written report. Although most presented reports a precious few tried original approaches, such as wall charts, cartoons, a class presentation with Q&A, and so on. Most importantly simply having the opportunity to present finished written work in ways that they determined meant that they thought about various ways in which to explain their ideas.

Here’s what I like about this concept and what I wish I had done in that class: I wish I had actually tapped into and understood the personal desires of my learners and showed them how to “play” with the learning requirements and how to best meet them in their work. I wish that I had encouraged them to produce artifacts that were original — that were personalized to their desires — that would meet the learning outcomes. And I wish that I had understood that my “failing” students — those that I unfairly deemed flakey, or lazy, or poor writers — probably did want to learn, but maybe not the way I thought they should. I wish … I wish … I wish…

This is why I am excited about heutagogy and wanted to start a Community of Practice focusing on it — it so nicely grounds and frames the approaches I try to use when I teach in a way that does not leave me wishing. It provides a way for me to organize and re-vision and re-enact my knowledge about self-directed learning; about authentic assessment; about learning outcomes, course design, and curricular structure; and about my preferred “teaching” approach.

I am so grateful to continue to learn about my practice — and to self-determine my path. Thanks to all who’ve joined the Heutagogy Community of Practice for already furthering my learning in ways I hadn’t ever anticipated.

Talking to Fish: From Packaging to Scaffolding (The Higher Ed Edition)

In our Heutagogy Community of Practice this past week, a link to a blog post by Jane Hart was shared as an example of a heutagogical practice in the workplace of the future (a side note: the future is actually now). Here’s the link:

The changing role of L & D: from “packaging” to “scaffolding”plus “social capability building”

Here’s a snippet of that post:

. . . how the future is about moving on from a focus on organizing others’ learning by “packaging” up lots of content, delivering it to them “on a plate”, and then managing access to it all.

Rather the future is going to be more about “scaffolding“.  I mean by this, working in partnership with the relevant team or group in the organization to help to provide a framework – ie the infrastructure (platforms, tools etc) as well as the right conditions for learning and performance support and improvement to take place.

And furthermore, rather than trying to design, create, deliver or even “control” what happens there, there is also a need for a focus on “building the new personal and social capabilities” that are are going to be required by the new “connected workers”, in order for them to work and learn effectively in the digitally connected workplace

The framework in that post has me thinking about its applicability to higher education. I think higher education has gotten really good at packaging: we package courses, degrees,  majors, and now MOOCs (of the “x” kind particularly) are just really big packages  (as if selling education in bulk at Costco). We package all of this stuff in time-based wrappers — credit hours, quarters/semesters, and seat time — with diplomas as the pretty bows on the top. And while there’s also some interesting un-packaging going on (think competency based education; think Prior Learning Assessment), and some fabulous teaching/learning/assessment practices within the packages, I think we still get stuck in the paradigm and our belief systems. And then it’s like talking to a fish about water: as fish, it’s hard to see our belief systems are just belief systems and ways of doing things because that’s just how we do things.

Thanks to lemonhalf on Flickr for making this image available for use.

Thanks to lemonhalf on Flickr for making this image available for use.

But what if higher education challenged this package model of content-wrapped-in-time and moved toward providing different infrastructures and conditions for learning, and also helped people get networked and connected in their learning (digitally, and in other ways)? What if we aimed for the 3rd and 4th rows of Jane Hart’s diagram, below?

How might this apply to higher education? What might it look like?

Jane Hart's diagram for moving from packaging to scaffolding and social capability building.

Jane Hart’s diagram for moving from packaging to scaffolding and social capability building.

I’m not at all suggesting there’s not a place for content in higher education — learning is usually about learning something, and why wouldn’t these somethings be organized into categories (dare I say more packages) we call academic disciplines?  I recently read my colleague’s blog post about the Khan Academy on Rauschenberg’s Bed and was once again reminded that content, and content expertise, matters.

But I also like to spend my time thinking about if there might just be a different way to go about it all.

Like talking to a fish about water: how would we know something could be different unless we jumped out of the pond once in a while to see what might be out there? It will certainly be different, and it just also might be better.

The Sky Is The Limit: MOOCs For Credit Or Learning?

Pam Tate from CAEL recently published this editorial in InsideHigherEd: The right path to MOOC credit?

Here are some essential questions she posed:

Are course evaluations and testing really the best or only way to deal with this new era of learning? What about experiential learning? If someone has college-level learning from their life experience is it invalid unless they take a course?

Tate proposes that  course-by-course  assessment may not be the best way to go, and that individual learning assessment via a portfolio may be a better approach. I couldn’t agree more, but I suspect our reasons are slightly different. My reason is about learning.

In the Heutagogy Community of Practice right now, we’re having a great conversation about the differences between learning, and between knowledge and skills acquisition. I think this is key to consider in the MOOC conversation, too. What kinds of learning are we interested in promoting in higher education? Surface, strategic, or deep? And then how do we design learning experiences to support the kinds of learning we really want?

Personally, I think the sky is the limit when we are talking about opening up access, in the way that MOOCs might, to learning. So why wouldn’t we go all the way?

The Sky is the Limit, photo by Harriet L. Schwartz with permission

The Sky is the Limit, photo by Harriet L. Schwartz with permission

MOOCs (specifically xMOOCs ) still seem to be all about delivering knowledge and skills acquisition; furthermore, now the hot topic is finding ways to “measure and credit” that learning (surface and strategic learning, but probably not deep learning), and ACE is doing it with the good old 20th century final exam. (Twelve steps forward for learning; 20 steps backwards for assessment!)

There’s a place for all three kinds of learning in our lives, of course, but if we leave deep learning development out of higher education, I think we miss a great collective opportunity, one that’s actually necessary for our modern global and technological society. As I often hear my colleague saying, I’m in higher education because ultimately I want to promote world peace. We can get closer to that via deep learning approaches. And deep learning approaches call for a different kind of assessment to support them.

This is why I’d much rather see a PLA portfolio-type process (reflect and integrate; learn and unlearn and relearn; articulate; clarify; analyze; identify significance; identify dissonance; MAKE MEANING!!! — these are key PLA verbs) for MOOC learning assessments than a final exam. A PLA-type process can deepen students’ learning  — can help learners construct knowledge and apply it to new situations, versus take it in and spit it out. In this way, assessment of learning  (prior or new, regardless of source) can serve to both measure and credit learning (assessment OF learning), but more importantly, to deepen learning and promote ongoing learning (assessment AS and FOR learning).

The sky is the limit when we are talking about opening up access to learning via MOOCs and other open resources. Why would we even think of stopping at the bottom rung?

About To Launch!

“Heutagogy is the study of self-determined learning … It is also an attempt to challenge some ideas about teaching and learning that still prevail in teacher centred learning and the need for, as Bill Ford (1997) eloquently puts it ‘knowledge sharing’ rather than ‘knowledge hoarding’. In this respect heutagogy looks to the future in which knowing how to learn will be a fundamental skill given the pace of innovation and the changing structure of communities and workplaces. “

Hase, S. and Kenyon, C. (2000). From andragogy to heutagogy. Ultibase, RMIT.  http://ultibase.rmit.edu.au/Articles/dec00/hase2.htm

____________________________________________________________________________

I’ve been working at this project since my sabbatical last summer, and it’s finally about to launch! We’re just waiting on the founders — Chris Kenyon and Stewart Chase — to send us the first Perspectives post and then we’re off!

Heutagogy Community of Practice

Please join us!

Post-Traditional Learners and Prior Learning Assessment: The Practice of Learning

They used to be called “adult learners;” a recent report by the American Council on Education (written by Louis Soares) calls them post-traditional learners:

Post-traditional learners have been a growing presence in America’s postsecondary education institutions since the late 1970s. In fact, by many measures these “non-traditional” students have become the norm in postsecondary education. But post-traditional learners are a diverse group. The term encompasses individuals with a range of education needs from high school graduates to high school dropouts and those with limited literacy and English language skills. Post-traditional learn- ers also encompass many life stages and identities; they are single mothers, immigrants, veterans, and at-risk younger people looking for a second chance. (page 2)

No doubt the ACE report is a bit self-serving; ACE has a lot to gain in helping post-traditional learners complete degrees. Here’s what the report says about Prior Learning Assessment:

While the MOOC discussion is inspiring excitement and trepidation, the important element to consider is the emergence of a set of entities which are capable of evaluating different learning experiences for credit-worthiness. Thus far this competency has been largely used at the margins of postsecondary education, not surprisingly because it is closely associated with the characteristics of post-traditional learners. MOOC initiatives could be the accelerant that moves these organizations to scale in mainstream postsecondary education delivery.

The investment of $500 billion in education outside the academy, the rise of corporate universities, and the expanded interest in prior learning assessment are all pointing to the emergence of an ecosystem for validating learning that encompasses and supersedes the academy. (page 11)

In this post-traditional world, I would like to see PLA be an integrative force for past, present, and future learning experiences regardless of source, not just “prior” learning. I would like to see PLA less as a validating process and more of a “meaning-making” and “making learning visible” process. I would like to see PLA be a heutagogical practice (I believe many PLA programs already are), facilitating double-loop learning and self-determined learning, and supporting the development of capable and competent learners!

The combination of praxis and reflection on experience and learning is powerful and significant for post-traditional learners in a post-traditional world defined by constant change. In this vein, PLA wouldn’t even be called PLA — it would be simply be called “the practice of learning.”