Earlier this week, an essay written by Adam Kotsko was published in InsideHigherEd called “Making the Best of Assessment.” Two key sections of his piece stood out to me:
Would I really object if someone suggested that my institution might want to clarify its goals, gather information about how it’s doing in meeting those goals, and change its practices if they are not working? I doubt that I would: in a certain sense it’s what every institution should be doing. Doing so systematically does bear significant costs in terms of time and energy — but then so does plugging away at something that’s not working. Paying a reasonable number of hours up front in the form of data collection seems like a reasonable hedge against wasting time on efforts or approaches that don’t contribute to our mission. By the same token, getting into the habit of explaining why we’re doing what we’re doing can help us to avoid making decisions based on institutional inertia.
Despite that overall optimism, however, I’m also sure that there are some things that we’re doing that aren’t working as well as they could, but we have no way of really knowing that currently. We all have limited energy and time, and so anything that can help us make sure we’re devoting our energy to things that are actually beneficial seems all to the good.
Recently, I experienced an assessment phenomenon that I’ve started to affectionately call “when data challenges our belief system.” (My colleague has also written about this here; when I was telling another colleague about this, she referred to it as “when reality interferes with our denial.”) I’ve been helping my colleagues work with assessment methods and their findings for many years now, but it only became apparent to me recently that sometimes we don’t want to believe what we see in our assessment findings or in other sources of data. Sometimes we’d prefer to just erase the evidence or the findings of a inquiry project rather than face the reality that we might be able to do something better, something different.
Joan Didion wrote a book with the title We tell ourselves stories in order to live.
In trying to build an evidence-informed culture for improvement in higher education, I have come to believe that our willingness to interrogate the stories we tell ourselves might be one of the biggest challenges we face. Resources? Yep: we certainly need those! Buy-in that the process is worthwhile? Totally important! Support and learning to enact effective assessment practices? Absolutely necessary. But …
We tell ourselves stories.
And guess what! I am totally guilty of this! Here’s a recent example: Using findings from a database-informed report I received almost 7 years ago, I believed that 95% of students who took an introductory course offered in my department went on to take many more courses at the university. This became my department’s story (because it was true — in 2006). It was a great story, until it wasn’t. When I requested and received an updated report, this is what we found out, and what we did:
Of 388 students from Summer 2009 – Fall 2013, 90 took only this course. Thus, 23% of students who took the course didn’t take anything else. And although this means 77% of students did take other courses (we can celebrate that – it could have been worse, after all), we needed to think about whether or not our original story (remember it? 95%?) still held water. My departmental colleagues and I discussed this report, and once we came to grips with the new story the data was telling us, we realized that we wanted to make some advising process improvements with the goal of increasing the number of students who go on to take more courses at the university.
What’s challenging from an assessment perspective is that this practice of storytelling can totally limit our ability to use information to make improvements; more significantly, it can limit our learning. Of course we should be analytical about our findings; we should understand their limitations, reliability and validity, the circumstances, the context. But if we believe everything to be perfectly fine — or even quite good — reliable findings that tell a different story can be hard to stomach. Sometimes what happens is folks blame the data; worse is that they blame the messenger; far, far worse is that they blame students.
We tell ourselves stories.
One of the greatest powers of assessment and of an evidence-informed, improvement-oriented culture is that it can foster critical reflection on practice, but only if we can be — if we’re willing to be — critically reflective. I also think assessment can foster really important conversations about students’ experiences and learning among colleagues (such as the great conversation we had in our department when the new data no longer supported our outdated story). In other words: assessment itself can foster our learning. But when we get stuck with our stories and we can’t see that there might be a different reality that’s out there, we shut learning out, and we shut out the opportunities that can result from learning.
Me thinking about thinking about heutagogy with a few others who are also thinking about it. It’s nice to think, together.
Originally posted on Heutagogy Community of Practice:
A few people have been thinking about heutagogy or heutagogical practices recently here in WordPress land, so let’s repost a few of these ideas here for everyone to read:
- Bernard Bull wrote this post on his site Etale – Life in the Digital World: A Primer on Three “gogies” #pedagogy #heutagogy #andragogy
- Jackie Gerston wrote this post on her site User Generated Education: Learners Should Be Developing Their Own Questions
- Helen Crump wrote this post on her site Learningcreep: POT Cert Week 19: promoting self-determination with web-enhanced teaching and learning
- Melanie Booth wrote this post on her site PrattleNog: Moving Past Wishes
Originally posted on Heutagogy Community of Practice:
Heutagogy: Reconceptualising Learning for the 21st Century – Stewart Hase
Since its inception in 2000, heutagogy or self-determined learning, has been discussed and applied in a number of different settings across the globe. This keynote address explores the fundamentals of heutagogy and its theoretical underpinnings. We will also examine a long overdue reconceptualization of learning based on advances in neuroscience and how this fits within heutagogy. Finally we will review recent applications. Consistent with the principles of heutagogy the session will be interactive involving a live Twitter feed and group discussion. To gain maximum benefit participants are requested to read about heutagogy by accessing the various papers and commentary that can be found on the web and particularly the heutagogy community of practice at http://heutagogycop.wordpress.com/. Participants can also contact Stewart at firstname.lastname@example.org.
I have about 50 posts started, but none finished — or at least none worthy to post. As usual, I’ve been prattling (though only in my head, so far) about change in higher education; about the importance of leaders-as-learners and teachers; about technology, learning, assessment, teaching; about heutagogy; about access; etc. etc. etc.
So many posts; so little posting.
And I did take this picture of a lovely spring flower while out on a bike ride with my kid:
So there’s that.
Instead of posting my own ramblings right now, I’ll share a few links to others’ pieces that have really inspired my nog as of late. I’ll also share a few quotes from each that I mentally highlighted – may they inspire you to inquire and reflect, too.
How Disruptive Is Information Technology Really? by Judith Ramaley – EDUCAUSE Review
The act of teaching is becoming more about designing the educational context and engaging students as they learn to approach material in more insightful and demanding ways. We are not transmitters of knowledge very often today, although an occasional superb lecture by a remarkably perceptive and even prescient speaker or a carefully crafted blog contribution can open up new ways of thinking about things.
We all know these simple things about how the educational experience is changing, but how recently have we paused to think about how truly wonderful it is to be able to use our smartphones to answer a question right immediately? My real concern is that not all questions have a quick, well-researched, and easy-to-find answer. Many, perhaps most, questions in today’s world are hard to formulate, are seen in very different ways by different people, or simply do not have good answers at all. That is why we still need real people who interact with each other in real time in order to frame questions that matter, to explore the ideas that come from those questions, and to work together to find solutions. No longer, however, are those people confined to the knowledge and experience that they carry in their own minds or that they can bring along with them on paper. They can tap into a true universe of material whenever they wish.
When I think about the role of what we traditionally think of as teacher, one of the most important roles of teachers is to work hard at making themselves as unimportant as possible, not unimportant in the sense of lacking value, but unimportant in the sense that they are eventually no longer needed. In other words, the goal of the teacher is to aid the learners in becoming self-directed learners.
Training and instruction are all about control, with curricula, sanctioned learning objectives, and performance criteria. This works when the field of study is knowable. But fewer fields remain completely knowable, if they ever were. Many institutions and professions have been built on the premise that knowledge can be transferred in some kind of controlled process. If you question that premise, you threaten people’s jobs, status, and sense of worth. This is why you see some violent reactions to the notion of informal and social learning having validity within organizations.
A major difference between communities of practice and work teams is that the former are voluntary. People want to join communities of practice. People feel affinity for their communities of practice. You know you are in a community of practice when it changes your practice.
And finally, a post from the Heutagogy Community of Practice: The PAH Continuum: Pedagogy, Andragogy, & Heutagogy, by Fred Garnett
When we collaboratively developed the ideas of the open context model of learning, Wilma Clark had pointed out that in Russia the word ‘obuchenie’ means both teaching and learning, and the PAH Continuum might be seen as a way of scaffolding ‘obuchenie’ as a move from teacher’s control to learner’s control. I would see it as axiomatic, as I did when I was ‘brokering’ learning, that teachers, whilst delivering their subject expertise, should be enabling learners to better understand the process of learning for themselves.
So many posts; so much learning. With great appreciation for all my teachers out there. As Bernard Bull wrote:
In one sense, a teacher is anyone or anything that contributes to our learning.
1. My son Mac, now 5, has been interested in the Titanic story for about a year now. I shared with him the theory that had the Titanic turned itself around and parked itself next to the iceberg that it had hit, most of the passengers and crew could have gotten off the boat, onto the berg, and been rescued. (According to some oceanic archeologists, the ship could have turned itself around.) They would have been cold, for sure, but they wouldn’t have drowned. Instead, they all saw the iceberg as the threat (The Threat!!!), so they steered away from it. They didn’t see the threat as what it could be — a life saver. (I can no find no reference to this now; I heard it many years ago so I imagine it’s totally urban myth. But for this story, it serves a purpose.)
2. I recently read a publication called An avalanche is coming: Higher education and the revolution ahead. If you work in higher education, you might consider reading it, or at least skimming it. The authors explain the avalanche metaphor this way: “The avalanche metaphor is appropriate because the one certainty for anyone in the path of an avalanche is that standing still is not an option. Indeed, it is a classic error of strategy to calculate the risks of action but fail to calculate the (often greater) risks of doing nothing” (pgs. 3-4).
I read words like “avalanche” (which sounds strong, forceful, and terrifying) and “revolution” (which sounds challenging and yet potentially transformative), and I get … I get … (what do I get?) Well, I get disconcerted. But I also am inspired by a key idea in this report:
Given the state of the global economy, tensions in international relations, massive gaps between wealth and poverty, the deepening threat of climate change and the ubiquity of weapons of mass destruction, our contention is that we need a generation better educated, in the broadest and most profound sense of that word, than ever before. We need – as the London 2012 Olympics promised– an inspired generation, all of whom are well-educated and some of whom are able to provide the bold, sophisticated leadership that the 21st century demands. We need citizens ready to take personal responsibility both for themselves and for the world around them: citizens who have, and seize, the opportunity to learn and relearn throughout their lives. We need citizens who are ready and able to take their knowledge of the best that has been thought and said and done and apply it to the problems of the present and the future. This surely should be the mission of universities, and here in An Avalanche is Coming we have sought to describe the threat posed to traditional 20th century universities if key institutions don’t change radically, as well as the huge opportunities open to them if they do.
Correction: Especially as applied to my real life.
Change motivates and challenges. It makes clear when things are obsolete or have outlived their usefulness. But most of all, change forces us to learn differently. If the twentieth century was about creating a sense of stability to buttress against change and then trying to adapt to it, then the twenty-first century is about embracing change, not fighting it. Embracing change means looking forward to what will come next. It means viewing the future as a new set of possibilities, rather than something that forces us to adjust. It means making the most of living in a world of motion. ~Douglas Thomas and John Seely Brown: A New Culture of Learning: Cultivating the Imagination for a World of Constant Change
4. Another idea that’s rattling in my head right now is this, but I don’t know whom to attribute it to other than a colleague who quipped it recently in a meeting:
Never waste a crisis.
“Crisis.” Yet another scary word that is being used quite a bit in higher ed: student debt crisis; academic quality crisis; online learning / MOOC crisis; full-time faculty crisis; part-time faculty crisis; crisis this and crisis that. You name it, it’s a crisis (if you decide to call it one).
Define “crisis,” higher ed. Is it crisis because it’s change? I seriously doubt it. Perhaps it’s a crisis because change challenges our beliefs about what higher education / our university / our program should be. And because we now have to think, and act, and learn (!) differently. Perhaps *this* is the avalanche, and the revolution — the avalanche is within us, not happening to us; and the revolution can result in freeing our thinking.
5. Mac told me this last night as I turned out his light after we read a book about the Titanic:
Mama, do you know what happens to an iceberg when it melts? It becomes water. It becomes something really different. It doesn’t get to do that if it stays an iceberg.
Icebergs, avalanches, revolutions, crises. These all demonstrate the risk of doing nothing: the most significant of which, as I see it, is not getting to learn, differently.
You’ll excuse me now please: I have to go do something.
One – Scarlett Johansson can sing. Listen to my new favorite song, Before My Time, from the documentary film Chasing Ice:
Two – This lovely animated cycling film. Oh my gosh: it’s just beautiful!
Many many many moons ago, when I was teaching English in my first “real” job in a community college in Washington state, I decided to try a technique called “negotiated assessment” that I’d heard about at a conference. The conference was about learning outcomes assessment — a fairly new paradigm at the time — and when I heard this idea I immediately wanted to try it. Why should I be the only one who assessed my students’ learning; why couldn’t they do it? Wouldn’t that help them pay more attention to their learning process and outcomes; wouldn’t it also, perhaps, potentially, lighten the power I wielded with my pen and my gradebook?
At the time it was a risky experiment for me: I didn’t yet have tenure, and here I was, giving the power of grading over to my students (except that I wasn’t — more about that below). My department chair (who was also the chair of my tenure committee) was supportive of the experiment, and so I tried it out. This was long before I knew much about learning or assessment; long before I knew how to support students in taking more ownership over their learning; long before I formally knew much about reflection, about deep learning, about metacognition, about self-directed and self-determined learning; long before I had devoured Freire and hooks and Dewey and Kegan and Daloz and Palmer and Doll and so many others; and it was long before I had enough experience to really design the approach so it worked well.
Nonetheless, here’s how it went in my English 101 – College Composition course that spring semester:
- I created the course description, the learning outcomes (some of which were departmentally prescribed), and the learning activities and assignments for the course.
- I led and facilitated the course activities and provided direct instruction when needed.
- Students engaged (or not, in some cases) in the writing processes I set up (and required), including peer review, responding to feedback on multiple drafts from me and other students, learning about revision as “re-seeing” and editing as cleaning up fly specks on the window so readers could see the message clearly, etc.
- I never graded a single paper; I provided LOTS of feedback on LOTS of drafts, but I never assigned an actual letter grade.
- At the end of the term, students wrote a final essay self-assessing their learning through the lens of the learning outcomes of the course and feedback they received from me and others along the way. They put the paper in the front of a binder that contained all of their other drafts and final papers (a very fundamental form of portfolio at the time). Based on their “analysis” and the “evidence of learning” in the binder, they recommended a final grade for themselves.
- The last week of the term, I met with each student individually to review the paper, the portfolio, and to discuss the grade they thought they should get, and to talk about how they should transfer their learning from this class to other classes and contexts (because surely, if I told them to, they would, right?)
It was a fascinating experiment in all sorts of things — students whom I considered to be “strong” graded themselves much harder than I would have; students I considered to be “weak” (or “flakey”) recommended A grades for themselves. (I am now ashamed that I had those thoughts about my students at that time, but I did.) During the meeting, we “negotiated” the student’s grade and came to agreement, using the portfolio as an anchor, on what that grade should be. (In actual fact I still actually did hold most of the power. For one thing, I totally over-ruled 2 of my 20 students. They most certainly did not earn an A let alone demonstrate much learning. What kind of pushover did they think I was!?!)
I was reminded of this experiment from many moons ago this past week when I read a new perspectives post about heutagogy contributed by Fred Garnett in the Heutagogy Community of Practice. Read it here:
In this post, Fred describes his practice of “brokering:”
Writing the syllabus and developing the schedule of delivery along with the work to be completed meant that I was, in effect, building the framework of what I was teaching. Consequently I really understood what the boundaries were and so could better broker between the formal requirements of the education system and the personal desires of my learners; I had found that all these ‘failing’ students wanted to learn. On the social impact course each student picked any technology that interested them to research and write about. I showed them how to “play” with the learning requirements, which can be used as creative constraints, and how best to meet them in their completed work. I also encouraged them to present that work in original ways rather than as just a written report. Although most presented reports a precious few tried original approaches, such as wall charts, cartoons, a class presentation with Q&A, and so on. Most importantly simply having the opportunity to present finished written work in ways that they determined meant that they thought about various ways in which to explain their ideas.
Here’s what I like about this concept and what I wish I had done in that class: I wish I had actually tapped into and understood the personal desires of my learners and showed them how to “play” with the learning requirements and how to best meet them in their work. I wish that I had encouraged them to produce artifacts that were original — that were personalized to their desires — that would meet the learning outcomes. And I wish that I had understood that my “failing” students — those that I unfairly deemed flakey, or lazy, or poor writers — probably did want to learn, but maybe not the way I thought they should. I wish … I wish … I wish…
This is why I am excited about heutagogy and wanted to start a Community of Practice focusing on it — it so nicely grounds and frames the approaches I try to use when I teach in a way that does not leave me wishing. It provides a way for me to organize and re-vision and re-enact my knowledge about self-directed learning; about authentic assessment; about learning outcomes, course design, and curricular structure; and about my preferred “teaching” approach.
I am so grateful to continue to learn about my practice — and to self-determine my path. Thanks to all who’ve joined the Heutagogy Community of Practice for already furthering my learning in ways I hadn’t ever anticipated.
In our Heutagogy Community of Practice this past week, a link to a blog post by Jane Hart was shared as an example of a heutagogical practice in the workplace of the future (a side note: the future is actually now). Here’s the link:
Here’s a snippet of that post:
. . . how the future is about moving on from a focus on organizing others’ learning by “packaging” up lots of content, delivering it to them “on a plate”, and then managing access to it all.
Rather the future is going to be more about “scaffolding“. I mean by this, working in partnership with the relevant team or group in the organization to help to provide a framework – ie the infrastructure (platforms, tools etc) as well as the right conditions for learning and performance support and improvement to take place.
And furthermore, rather than trying to design, create, deliver or even “control” what happens there, there is also a need for a focus on “building the new personal and social capabilities” that are are going to be required by the new “connected workers”, in order for them to work and learn effectively in the digitally connected workplace
The framework in that post has me thinking about its applicability to higher education. I think higher education has gotten really good at packaging: we package courses, degrees, majors, and now MOOCs (of the “x” kind particularly) are just really big packages (as if selling education in bulk at Costco). We package all of this stuff in time-based wrappers — credit hours, quarters/semesters, and seat time — with diplomas as the pretty bows on the top. And while there’s also some interesting un-packaging going on (think competency based education; think Prior Learning Assessment), and some fabulous teaching/learning/assessment practices within the packages, I think we still get stuck in the paradigm and our belief systems. And then it’s like talking to a fish about water: as fish, it’s hard to see our belief systems are just belief systems and ways of doing things because that’s just how we do things.
But what if higher education challenged this package model of content-wrapped-in-time and moved toward providing different infrastructures and conditions for learning, and also helped people get networked and connected in their learning (digitally, and in other ways)? What if we aimed for the 3rd and 4th rows of Jane Hart’s diagram, below?
How might this apply to higher education? What might it look like?
I’m not at all suggesting there’s not a place for content in higher education — learning is usually about learning something, and why wouldn’t these somethings be organized into categories (dare I say more packages) we call academic disciplines? I recently read my colleague’s blog post about the Khan Academy on Rauschenberg’s Bed and was once again reminded that content, and content expertise, matters.
But I also like to spend my time thinking about if there might just be a different way to go about it all.
Like talking to a fish about water: how would we know something could be different unless we jumped out of the pond once in a while to see what might be out there? It will certainly be different, and it just also might be better.
Here are some essential questions she posed:
Are course evaluations and testing really the best or only way to deal with this new era of learning? What about experiential learning? If someone has college-level learning from their life experience is it invalid unless they take a course?
Tate proposes that course-by-course assessment may not be the best way to go, and that individual learning assessment via a portfolio may be a better approach. I couldn’t agree more, but I suspect our reasons are slightly different. My reason is about learning.
In the Heutagogy Community of Practice right now, we’re having a great conversation about the differences between learning, and between knowledge and skills acquisition. I think this is key to consider in the MOOC conversation, too. What kinds of learning are we interested in promoting in higher education? Surface, strategic, or deep? And then how do we design learning experiences to support the kinds of learning we really want?
Personally, I think the sky is the limit when we are talking about opening up access, in the way that MOOCs might, to learning. So why wouldn’t we go all the way?
MOOCs (specifically xMOOCs ) still seem to be all about delivering knowledge and skills acquisition; furthermore, now the hot topic is finding ways to “measure and credit” that learning (surface and strategic learning, but probably not deep learning), and ACE is doing it with the good old 20th century final exam. (Twelve steps forward for learning; 20 steps backwards for assessment!)
There’s a place for all three kinds of learning in our lives, of course, but if we leave deep learning development out of higher education, I think we miss a great collective opportunity, one that’s actually necessary for our modern global and technological society. As I often hear my colleague saying, I’m in higher education because ultimately I want to promote world peace. We can get closer to that via deep learning approaches. And deep learning approaches call for a different kind of assessment to support them.
This is why I’d much rather see a PLA portfolio-type process (reflect and integrate; learn and unlearn and relearn; articulate; clarify; analyze; identify significance; identify dissonance; MAKE MEANING!!! — these are key PLA verbs) for MOOC learning assessments than a final exam. A PLA-type process can deepen students’ learning – can help learners construct knowledge and apply it to new situations, versus take it in and spit it out. In this way, assessment of learning (prior or new, regardless of source) can serve to both measure and credit learning (assessment OF learning), but more importantly, to deepen learning and promote ongoing learning (assessment AS and FOR learning).
The sky is the limit when we are talking about opening up access to learning via MOOCs and other open resources. Why would we even think of stopping at the bottom rung?