We Tell Ourselves Stories

Earlier this week, an essay written by Adam Kotsko was published in InsideHigherEd called “Making the Best of Assessment.” Two key sections of his piece stood out to me:

Would I really object if someone suggested that my institution might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if they are not working? I doubt that I would: in a certain sense it’s what every institution should be doing. Doing so systematically does bear significant costs in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in the form of data collection seems like a reasonable hedge against wasting time on efforts or approaches that don’t contribute to our mission. By the same token, getting into the habit of explaining why we’re doing what we’re doing can help us to avoid making decisions based on institutional inertia.

And a bit later on in the piece, referring to his own institution, department, and colleagues:
Despite that overall optimism, however, I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently. We all have limited energy and time, and so anything that can help us make sure we’re devoting our energy to things that are actually beneficial seems all to the good.

Recently, I experienced an assessment phenomenon that I’ve started to affectionately call “when data challenges our belief system.” (My colleague has also written about this here; when I was telling another colleague about this, she referred to it as “when reality interferes with our denial.”) I’ve been helping my colleagues work with assessment methods and their findings for many years now, but it only became apparent to me recently that sometimes we don’t want to believe what we see in our assessment findings or in other sources of data. Sometimes we’d prefer to just erase the evidence or the findings of a inquiry project rather than face the reality that we might be able to do something better, something different.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Thanks to F. Jourde on Flickr for making this image available for use through a Creative Commons license.

Joan Didion wrote a book with the title We tell ourselves stories in order to live.

In trying to build an evidence-informed culture for improvement in higher education, I have come to believe that our willingness to interrogate the stories we tell ourselves might be one of the biggest challenges we face. Resources? Yep: we certainly need those! Buy-in that the process is worthwhile? Totally important! Support and learning to enact effective assessment practices? Absolutely necessary. But …

We tell ourselves stories.

And guess what! I am totally guilty of this! Here’s a recent example: Using findings from a database-informed report I received almost 7 years ago, I believed that 95% of students who took an introductory course offered in my department went on to take many more courses at the university. This became my department’s story (because it was true — in 2006). It was a great story, until it wasn’t. When I requested and received an updated report, this is what we found out, and what we did:

Of 388 students from Summer 2009 – Fall 2013, 90 took only this course.  Thus, 23% of students who took the course didn’t take anything else. And although this means 77% of students did take other courses (we can celebrate that – it could have been worse, after all), we needed to think about whether or not our original story (remember it? 95%?) still held water. My departmental colleagues and I discussed this report, and once we came to grips with the new story the data was telling us, we realized that we wanted to make some advising process improvements with the goal of increasing the number of students who go on to take more courses at the university.

What’s challenging from an assessment perspective is that this practice of storytelling can totally limit our ability to use information to make improvements; more significantly, it can limit our learning.  Of course we should be analytical about our findings; we should understand their limitations, reliability and validity, the circumstances, the context. But if we believe everything to be perfectly fine — or even quite good — reliable findings that tell a different story can be hard to stomach. Sometimes what happens is folks blame the data; worse is that they blame the messenger; far, far worse is that they blame students.

We tell ourselves stories.

One of the greatest powers of assessment and of an evidence-informed, improvement-oriented culture is that it can foster critical reflection on practice, but only if we can be — if we’re willing to be — critically reflective. I also think assessment can foster really important conversations about students’ experiences and learning among colleagues (such as the great conversation we had in our department when the new data no longer supported our outdated story).  In other words: assessment itself can foster our learning.  But when we get stuck with our stories and we can’t see that there might be a different reality that’s out there, we shut learning out, and we shut out the opportunities that can result from learning.

No doubt that stories are important — to culture, to humanity, to life. As Brene Brown said in this TED talk, “stories are data with soul.”  I do think our stories can inform us and teach us (hey – I was an English major; of course I think this!) But I also think that if we’re not open to alternative views, to other voices (namely those of students), and to new information, then we’re being irresponsible educators because we’re not walking our talk. And so I’ve come to realize that a big part of my work is to support people in being willing to engage in critical reflection on practice, and to consider changing their stories if needed using the information they learn from assessment (it may come as no surprise to some of you that my work with Prior Learning Assessment and interest in heutagogy greatly inform my perspective and my approach). After all, education is supposed to be about learning; let’s use information we have to learn, and to get better.

So Many Posts; So Little … Posting

I have about 50 posts started, but none finished — or at least none worthy to post. As usual, I’ve been prattling (though only in my head, so far) about change in higher education; about the importance of leaders-as-learners and teachers; about technology, learning, assessment, teaching; about heutagogy; about access; etc. etc. etc.

So many posts; so little posting.

While not posting, I have been contemplating the lessons in two poems by Marge Piercy: The Seven of Pentacles and To Be of Use.

And I did take this picture of a lovely spring flower while out on a bike ride with my kid:

Meditation on a spring flower in the forest.

Meditation on a spring flower in the forest.

So there’s that.

Instead of posting my own ramblings right now, I’ll share a few links to others’ pieces that have really inspired my nog as of late.  I’ll also share a few quotes from each that I mentally highlighted – may they inspire you to inquire and reflect, too.

How Disruptive Is Information Technology Really? by Judith Ramaley – EDUCAUSE Review

The act of teaching is becoming more about designing the educational context and engaging students as they learn to approach material in more insightful and demanding ways. We are not transmitters of knowledge very often today, although an occasional superb lecture by a remarkably perceptive and even prescient speaker or a carefully crafted blog contribution can open up new ways of thinking about things.

We all know these simple things about how the educational experience is changing, but how recently have we paused to think about how truly wonderful it is to be able to use our smartphones to answer a question right immediately? My real concern is that not all questions have a quick, well-researched, and easy-to-find answer. Many, perhaps most, questions in today’s world are hard to formulate, are seen in very different ways by different people, or simply do not have good answers at all. That is why we still need real people who interact with each other in real time in order to frame questions that matter, to explore the ideas that come from those questions, and to work together to find solutions. No longer, however, are those people confined to the knowledge and experience that they carry in their own minds or that they can bring along with them on paper. They can tap into a true universe of material whenever they wish.

Good Teachers Become Less Important by Bernard Bull – Etale – Life in the Digital World.

When I think about the role of what we traditionally think of as teacher, one of the most important roles of teachers is to work hard at making themselves as unimportant as possible, not unimportant in the sense of lacking value, but unimportant in the sense that they are eventually no longer needed.  In other words, the goal of the teacher is to aid the learners in becoming self-directed learners.

Notes on social learning in business by Harold Jarche

Training and instruction are all about control, with curricula, sanctioned learning objectives, and performance criteria. This works when the field of study is knowable. But fewer fields remain completely knowable, if they ever were. Many institutions and professions have been built on the premise that knowledge can be transferred in some kind of controlled process. If you question that premise, you threaten people’s jobs, status, and sense of worth. This is why you see some violent reactions to the notion of informal and social learning having validity within organizations.

A major difference between communities of practice and work teams is that the former are voluntary. People want to join communities of practice. People feel affinity for their communities of practice. You know you are in a community of practice when it changes your practice.

And finally, a post from the Heutagogy Community of Practice: The PAH Continuum: Pedagogy, Andragogy, & Heutagogy, by Fred Garnett

When we collaboratively developed the ideas of the open context model of learning, Wilma Clark had pointed out that in Russia the word ‘obuchenie’ means both teaching and learning, and the PAH Continuum might be seen as a way of scaffolding ‘obuchenie’ as a move from teacher’s control to learner’s control. I would see it as axiomatic, as I did when I was ‘brokering’ learning, that teachers, whilst delivering their subject expertise, should be enabling learners to better understand the process of learning for themselves.

So many posts; so much learning. With great appreciation for all my teachers out there. As Bernard Bull wrote:

In one sense, a teacher is anyone or anything that contributes to our learning.

Moving Past Wishes

Many many many moons ago, when I was teaching English in my first “real” job in a community college in Washington state, I decided to try a technique called “negotiated assessment” that I’d heard about at a conference. The conference was about learning outcomes assessment — a fairly new paradigm at the time — and when I heard this idea I immediately wanted to try it. Why should I be the only one who assessed my students’ learning; why couldn’t they do it? Wouldn’t that help them pay more attention to their learning process and outcomes; wouldn’t it also, perhaps, potentially, lighten the power I wielded with my pen and my gradebook?

Grade A

Many thanks to Steve Snodgrass on Flickr for making this image available to use.

At the time it was a risky experiment for me: I didn’t yet have tenure, and here I was, giving the power of grading over to my students (except that I wasn’t — more about that below). My department chair (who was also the chair of my tenure committee) was supportive of the experiment, and so I tried it out. This was long before I knew much about learning or assessment; long before I knew how to support students in taking more ownership over their learning; long before I formally knew much about reflection, about deep learning, about metacognition, about self-directed and self-determined learning; long before I had devoured Freire and hooks and Dewey and Kegan and Daloz and Palmer and Doll and so many others; and it was long before I had enough experience  to really design the approach so it worked well.

Nonetheless, here’s how it went in my English 101 – College Composition course that spring semester:

  • I created the course description, the learning outcomes (some of which were departmentally prescribed), and the learning activities and assignments for the course.
  • I led and facilitated the course activities and provided direct instruction when needed.
  • Students engaged (or not, in some cases) in the writing processes I set up (and required), including peer review, responding to feedback on multiple drafts from me and other students, learning about revision as “re-seeing” and editing as cleaning up fly specks on the window so readers could see the message clearly, etc.
  • I never graded a single paper; I provided LOTS of feedback on LOTS of drafts, but I never assigned an actual letter grade.
  • At the end of the term, students wrote a final essay self-assessing their learning through the lens of the learning outcomes of the course and feedback they received from me and others along the way. They put the paper in the front of a binder that contained all of their other drafts and final papers (a very fundamental form of portfolio at the time). Based on their “analysis” and the “evidence of learning” in the binder, they recommended a final grade for themselves.
  • The last week of the term, I met with each student individually to review the paper, the portfolio, and to discuss the grade they thought they should get, and to talk about how they should transfer their learning from this class to other classes and contexts (because surely, if I told them to, they would, right?)

It was a fascinating experiment in all sorts of things — students whom I considered to be “strong” graded themselves much harder than I would have; students I considered to be “weak” (or “flakey”) recommended A grades for themselves. (I am now ashamed that I had those thoughts about my students at that time, but I did.) During the meeting, we “negotiated” the student’s grade and came to agreement, using the portfolio as an anchor, on what that grade should be. (In actual fact I still actually did hold most of the power. For one thing, I totally over-ruled 2 of my 20 students. They most certainly did not earn an A let alone demonstrate much learning. What kind of pushover did they think I was!?!)

I was reminded of this experiment from many moons ago this past week when I read a new perspectives post about heutagogy contributed by Fred Garnett in the Heutagogy Community of Practice.  Read it here:

The PAH Continuum: Pedagogy, Andragogy & Heutagogy

In this post, Fred describes his practice of  “brokering:”

Writing the syllabus and developing the schedule of delivery along with the work to be completed meant that I was, in effect, building the framework of what I was teaching. Consequently I really understood what the boundaries were and so could better broker between the formal requirements of the education system and the personal desires of my learners; I had found that all these ‘failing’ students wanted to learn.  On the social impact course each student picked any technology that interested them to research and write about. I showed them how to “play” with the learning requirements, which can be used as creative constraints, and how best to meet them in their completed work. I also encouraged them to present that work in original ways rather than as just a written report. Although most presented reports a precious few tried original approaches, such as wall charts, cartoons, a class presentation with Q&A, and so on. Most importantly simply having the opportunity to present finished written work in ways that they determined meant that they thought about various ways in which to explain their ideas.

Here’s what I like about this concept and what I wish I had done in that class: I wish I had actually tapped into and understood the personal desires of my learners and showed them how to “play” with the learning requirements and how to best meet them in their work. I wish that I had encouraged them to produce artifacts that were original — that were personalized to their desires — that would meet the learning outcomes. And I wish that I had understood that my “failing” students — those that I unfairly deemed flakey, or lazy, or poor writers — probably did want to learn, but maybe not the way I thought they should. I wish … I wish … I wish…

This is why I am excited about heutagogy and wanted to start a Community of Practice focusing on it — it so nicely grounds and frames the approaches I try to use when I teach in a way that does not leave me wishing. It provides a way for me to organize and re-vision and re-enact my knowledge about self-directed learning; about authentic assessment; about learning outcomes, course design, and curricular structure; and about my preferred “teaching” approach.

I am so grateful to continue to learn about my practice — and to self-determine my path. Thanks to all who’ve joined the Heutagogy Community of Practice for already furthering my learning in ways I hadn’t ever anticipated.

The Sky Is The Limit: MOOCs For Credit Or Learning?

Pam Tate from CAEL recently published this editorial in InsideHigherEd: The right path to MOOC credit?

Here are some essential questions she posed:

Are course evaluations and testing really the best or only way to deal with this new era of learning? What about experiential learning? If someone has college-level learning from their life experience is it invalid unless they take a course?

Tate proposes that  course-by-course  assessment may not be the best way to go, and that individual learning assessment via a portfolio may be a better approach. I couldn’t agree more, but I suspect our reasons are slightly different. My reason is about learning.

In the Heutagogy Community of Practice right now, we’re having a great conversation about the differences between learning, and between knowledge and skills acquisition. I think this is key to consider in the MOOC conversation, too. What kinds of learning are we interested in promoting in higher education? Surface, strategic, or deep? And then how do we design learning experiences to support the kinds of learning we really want?

Personally, I think the sky is the limit when we are talking about opening up access, in the way that MOOCs might, to learning. So why wouldn’t we go all the way?

The Sky is the Limit, photo by Harriet L. Schwartz with permission

The Sky is the Limit, photo by Harriet L. Schwartz with permission

MOOCs (specifically xMOOCs ) still seem to be all about delivering knowledge and skills acquisition; furthermore, now the hot topic is finding ways to “measure and credit” that learning (surface and strategic learning, but probably not deep learning), and ACE is doing it with the good old 20th century final exam. (Twelve steps forward for learning; 20 steps backwards for assessment!)

There’s a place for all three kinds of learning in our lives, of course, but if we leave deep learning development out of higher education, I think we miss a great collective opportunity, one that’s actually necessary for our modern global and technological society. As I often hear my colleague saying, I’m in higher education because ultimately I want to promote world peace. We can get closer to that via deep learning approaches. And deep learning approaches call for a different kind of assessment to support them.

This is why I’d much rather see a PLA portfolio-type process (reflect and integrate; learn and unlearn and relearn; articulate; clarify; analyze; identify significance; identify dissonance; MAKE MEANING!!! — these are key PLA verbs) for MOOC learning assessments than a final exam. A PLA-type process can deepen students’ learning  — can help learners construct knowledge and apply it to new situations, versus take it in and spit it out. In this way, assessment of learning  (prior or new, regardless of source) can serve to both measure and credit learning (assessment OF learning), but more importantly, to deepen learning and promote ongoing learning (assessment AS and FOR learning).

The sky is the limit when we are talking about opening up access to learning via MOOCs and other open resources. Why would we even think of stopping at the bottom rung?

Way-Finding, Delicacy, and Balance: The Oregon Edition

Thanks to lakewentworth on Flickr for making this Portland image available for use.

Thanks to lakewentworth on Flickr for making this Portland, Oregon image available for use.

Since October, I have been serving on the Credit for Prior Learning Advisory Committee of the State of Oregon Higher Education Coordinating Commission (wow – that’s a long title!). Dr. Larry Large of OAICU and I represent the independent colleges and universities, and we serve with colleagues representing community colleges, private for-profits, and the Oregon State University system.

So what’s this all about?

In short, in February 2012, the Oregon Legislature passed House Bill 4059 which “Directs  Higher  Education  Coordinating  Commission  to  carry  out  goals  relating  to  awarding  academic  credit  for  prior  learning  by  students.” The Bill has several goals, the two most significant being:

(a) Increase the number of students who receive academic credit for prior learning and the number of students who receive academic credit for prior learning that counts toward their major or toward earning their degree, certificate or credential, while ensuring that credit is awarded only for high quality course-level competencies;

(b) Increase the number and type of academic credits accepted for prior learning in institutions of higher education, while ensuring that credit is awarded only for high quality course-level competencies;

Read our 2012 report HERE.

As you can imagine, I am pretty excited about this Bill and the opportunities it will create to recognize and reward learning that students bring with them to college. I am also learning a lot — about the state legislative process, about people’s perceptions of CPL and PLA, and about what I need to advocate for (which I’ve written about ad nauseum).

Also, I recently reviewed the 2013 Horizon Report and several trends (starting on page 13) stood out to me as relevant to our CPL conversation, especially these:

  • The abundance of resources and relationships made easily accessible via the Internet is increasingly challenging us to revisit our roles as educators.
  • Assessment and accreditation are changing to validate life-long learning.
  • Both formal and informal learning experiences are becoming increasingly important as college graduates continue to face a highly competitive workforce.
  • Open is a key trend in future education and publication, specifically in terms of open content, open educational resources, massively open online courses, and open access.

CPL and PLA are going to be ever-important practices in the 21st century world of open learning, as Conrad proposes in this article, Assessment challenges in open learning: Way-finding, fork in the road, or end of the line?

This paper proposes that the adaptation of a rigorous RPL assessment process, modeled on some processes in operation at various post secondary institutions around the world, could offer a solution to the open learning assessment issue, a solution that would be academically viable, reputable, and sufficiently constructivist-oriented so as not to negate the energy and spirit already exercised by open learners. A delicate balancing act? Perhaps so, but in times of rapid, important, and disruptive change, both delicacy and the need for balancing abound. (p. 44)

Oregon is way-finding — and in doing this important work on behalf of Oregon learners, we need to remember that there already exist pedagogically sound CPL frameworks and practices that actually support deepening students’ learning. Icing on the credit cake. Let’s take a bite!

Convergence Zone: Heutagogy

I was a pop-in-and-out participant in last week’s MOOC MOOC, during which time I wrote this reflection about MOOCs, Prior Learning Assessment, and College-Level Learning. I read several tweets and posts from folks participating along the way that shaped and changed my thinking, and I gleaned a few interesting ideas and a whole lot of thoughtful questions about MOOCs, and even more thoughtful questions about teaching, learning, assessment, higher education, prior learning and assessment, pedagogy, technology, accreditation, adult learning, and the intersections of all of these topics.

Thanks to fabonthemoon on Flickr for making this image available for use!

This busy intersection is where I spend a lot of my time professionally and theoretically – smack dab in the middle of several noisy streets, trying to determine what way I might want to travel next before I get hit by oncoming traffic. It’s really not as terrifying as it might sound; in fact, the metaphor, for me, represents the complexity of higher education in a good way: its systems, its people, its opportunities and challenges. Traffic is moving here, and quickly, but then again, slowly. (It is higher education, after all.) 

One post that caught my attention was written by Dave Cormier titled Rhizomatic Learning and MOOCs – Assessment. Here’s a key point he made that’s been rattling in my nog since I read it:

IF

What we are learning is contextualized by each individual differently, according to their experiences, their understanding and purposes,

AND

The things that are learned are not definite, but flexible and complex

THEN

Assessing what someone ‘knows’ is an act of enforcement of a given point of view, not a(n apolotical) [sic] helpful guideline to learning

I want assessment to be a helpful guideline to learning (as reflected, for example, in my concerns with learning analytics). Period. With or without MOOCs, my personal vision of assessment’s main value is as a key part of an individual’s and group’s learning processes and cycles (assessment as an act of learning — for and as learning — versus of someone’s learning by someone else). I also think there is great value in assessment as and for learning for organizations. Otherwise, how would we know how we’re doing and how to improve?

Yet here’s where it gets messy:

When someone (presumably a person we call a “student”) wants to have their learning “certified” in some way and wants the currency of higher education — credits and degrees and whatnot — attached to that learning to “verify” that they have it, then these ideals of learning/assessment get messy and murky. (The same might be said of organizations: verification for higher educational organizations is just called “accountability” and is often conflated with “assessment” and “accreditation.”) For a student, passing a course somehow represents that he or she has learned something from it; adding up these courses to a degree presumes that he or she is qualified in particular ways. Somehow we have to “validate” all of this, for all sorts of reasons.

Prior Learning Assessment has been a way for students — typically adults with several years of life and professional experience under their belts — to have their learning “validated,” to earn credit for their prior college-level learning that they have obtained through non-college learning opportunities. By critically reflecting on their learning experiences, making meaning from these experiences, and connecting and integrating their own knowledge with that of the “academic experts,” students can demonstrate that they’ve met course learning outcomes through different sources other than the course. (GASP!) But perhaps the most meaningful aspect of PLA, in addition to acknowledging that people can learn all sorts of things in all sorts of ways from all sorts of sources, is that the process helps develop reflective thinking, awareness, problem-solving, and mindfulness about learning. In other words:

PLA helps learners be better learners!

So if PLA were to become a way to assess MOOC-based learning (which it could be) — if it were to be “re-branded,” as one person in my PLA LinkedIn group discussion forum suggested, to focus less on assessing prior, experiential learning for adults and to focus more on assessing college-level learning for all, then by all means, it could legitimately serve an assessment-of-MOOC-learning function (and frankly, it likely will).

But I keep coming back to what might be lost by using PLA in this way, for this purpose, and it comes down to my fear of losing the other 2 aspects of PLA: assessment for and as learning. Furthermore, I fear that there would be additional aspects to PLA that might be sacrificed or forgotten about. Here’s my quick and dirty laundry list:

  • The focus on the adult-as-learner specifically
  • Development of the critically reflective process required for turning experience into learning
  • The transformative (life-changing) outcomes of PLA as a learning process
  • The kind of thinking and problem-solving skills that folks develop when engage in PLA
  • The great sense of accomplishment that students feel and confidence that students gain when their learning is awarded college credit.

(See this PLA Bibliography for literature that reflects all of the above theoretically and in practice.)

I really don’t know much about MOOCs, or about rhizomatic learning, but I do know a few things about learning to learn, and I know a lot about PLA. I really would want to retain the kind of PLA that supports this metacognitive development in addition to validating someone’s experiential learning.

Now, this busy intersection is beginning to reveal a convergence zone for me: a focus on learning to learn, on developing capability as a learner — regardless of the topic or content area, the role or methods of assessment, who is teaching, or how or where one gains learning. This zone is called heutagogy, and I am working on creating a curation of resources and a community of practitioners to think together about its usefulness in this 21st century learning and higher educational landscape. I openly invite participation and perspectives: stayed tuned for more about Heutagogy In Action.

MORE ABOUT HEUTAGOGY

Blaschke, L.M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. International Review of Research in Open and Distance Learning, 13(1), 56-71. Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/1076/2113

Bhoyrub, J., Hurley, J., Neilson, G.R., Ramsay, M., & Smith, M. (2010). Heutagogy: An alternative practice based learning approachNurse Education in Practice19(6), 322-326.

Canning, N. &  Callan, S. (2010). Heutagogy: Spirals of reflection to empower learners in higher education. Reflective Practice, 11(1), pp. 71–82.

Canning, N. (2010). Playing with heutagogy: Exploring strategies to empower mature learners in higher education. Journal of Further and Higher Education, 34(1), pp. 59–71.

Chapnick, S. & Meloy, J. (2005).  Renaissance eLearning: creating dramatic and unconventional learning experiences. Essential resources for training and HR professionals. Pfeiffer: San Francisco, CA. (See “From Andragogy to Heutagogy,” pp. 36–39.)

Hase, S, and Kenyon, C. (2000). From andragogy to heutagogy. Ultibase, RMIThttp://ultibase.rmit.edu.au/Articles/dec00/hase2.htm

Hase, S, & Kenyon, C. (2007). Heutagogy: a child of complexity theory. Complicity: an International Journal of Complexity and Education, vol. 4, no. 1, pp. 111-118. “Heutagogy: A Child of Complexity Theory”

Hase, S, & Kenyon, C. (2001). Moving from Andragogy to Heutagogy in Vocational Education. Retrieved from: http://www.avetra.org.au/abstracts_and_papers_2001/Hase-Kenyon_full.pdf