So How, Exactly, Will Learning Be Assessed?

It is true that I flunked out of Girl Scouts long before I could get a vest like this.

I am trying to understand badges. Given my background with Prior Learning Assessment programs and other forms of experiential learning, I totally get the idea conceptually. (Hey – I know a few boy and girl scouts too! They’ve earned badges for things they learned and could do too, right?)

Apparently, with a badge:

You can get credit for learning outside of school, on the web, or from work and life experience.

So it sounds like some sort of technologically-mediated PLA, except the credit you earn is is not college credit. There is some other currency here, though what value it will hold remains to be seen.  It seems to me that there’s great potential here, as this article explains. What with the rising costs of higher education and access issues, and the learning opportunities presented with new technologies, combined with learning one can obtain through experience (hopefully with some expert guidance and reflection), this might be something quite worthwhile.  I can see it now: Post a few badges onto your robust ePortfolio and your avatar may actually scream: Hire Me!

But here’s what I don’t get — where is the “quality control?” (Ack – I hate that term applied to learning, but I think it’s a legitimate question).  My burning question with all of this is:

So, how, exactly, will learning be assessed?

Of course I ask this question of my faculty colleagues in higher education too — it’s certainly not a question specific to badges. I am honestly trying to figure it out!

Here is some information about assessment and badges, according to the Badge FAQs (a wiki, so what I’ve copied in blue will likely change soon):

Who can issue badges?
Badges can be created, defined and issued by a number of sources, including:

  • Traditional educational institutions (e.g., x, y or z)
  • Professional bodies (e.g. doctors, engineers, accountants)
  • International credential assessment agencies
  • Non formal, community learning organizations (e.g. Adult Basic Education, Literacy, Employability)
  • Communities of practice (e.g., open education projects, peer learners, or the individual learners themselves)
  • After-school programs and learning networks.
  • Online courses and open courseware initiatives.
  • Companies/organizations that employ people

How will the value of the badges be authenticated? In this system, a digital badge is more than just an image – it is essentially a collection of metadata that fully explains the badge and includes information such as the issuer, issue date, criteria for earning the badge, expiration if needed, the learner work or evidence behind the badge, etc. So the badge acts as a gateway or conversation starter, but the bulk of the information is in that metadata and it can act as an informal validation system itself.

And this is what the Mozilla and P2PU badge pilot project will address regarding assessment specifically (also from the FAQs):


The pilot will explore a range of assessment types, including:

  • peer assessment
  • self-assessment
  • portfolio assessment
  • stealth assessment
  • The Javascript badge assessment, for example, will require learners to submit work that demonstrates competency. Peers will then rate the work against a predefined rubric and set of criteria. Once the rating reaches a particular threshold, the badge will be issued.
  • The Accessibility badge will require experience designing or developing for challenged users or accessibility technologies, plus a blog post with reflection and analysis of the experience. A group of accessibility gurus within the community will then assess the work and issue badges accordingly.
  • Other badges may be aligned directly with courses, with course organizers able to assess work and issue badges.

And more:

How does assessment work?

  • For badges to hold real value and carry the weight of more traditional grades or degrees, assessment and quality is critical.
  • Badges can contain multiple levels of assessment, depending on the use case, community or intended audience
    • some will require distinct pre-defined assessment exercises and success criteria
    • others may be loosely defined and require learner reflection or peer recommendations.
  • Hard skills may require standard or more rigid rubrics to compare learner work against.
  • Softer skills can be more fluid and require more open and social assessments like peer reviews or endorsements.
  • For certification badges, intended for audiences like hiring managers, admission boards, more rigorous assessments can be required
  • For badges intended to simply build community or reward behaviors, simple assessments may be enough

How can badges provide greater flexibility and innovation in assessment?
Badges can help:

  • drive innovation around new types of assessments (e.g., x or y)
  • provide more personalized assessments for learners (e.g., x or y)
  • move beyond out of date or irrelevant testing practices (e.g., x or y)

For example:

  • Asynchronous assessment. Instead of being required to take an exam at a pre-determined time, for example, learners can seek out the assessment on their own time.
  • “Stealth assessment.” Assessment and awarding badges can happen automatically and provide immediate feedback. [Need a half sentence summary of what “stealth assessment is.]
  • Portfolio assessment. Work samples, projects and other artifacts the learner has produced or been involved in can demonstrate skills and competencies.
  • Multiple assessors or group assessment. In traditional classrooms, an individual instructor generally does most of the assessing. An open badge system can support assessment from multiple contexts, including course organizers, peers, or learners themselves. This flexible and networked nature could mean that there are multiple paths or assessment options for earning a badge, making the system more flexible, ensuring that the needs of each learner are met and limiting the learning path constraints.

I  think this is all very exciting. But I’ll need to keep learning about this badge idea and seeing where and how it goes because my jury is still out. But hey – maybe I can earn a badge for learning about badges. Then I can get an outfit for my avatar that looks something like this:

Just like I say about earning a college degree, maybe it’s never to late to earn a badge!


This Makes Us Official

Marylhurst University

My colleagues in the Assessment Program and I finally got our program description and content on the Marylhurst website. We’ve been wanting to do so for a very long time, but we needed the time and space to get it all organized. Thanks to the great work of our Educational Assessment Specialist Sione, we’re now present, virtually.

I think that makes us official now, right?

Check it out:  Assessment at Marylhurst University

Get Yourself A Blind Spot Prevention Team

I posted yesterday’s blog post and then sadly and belatedly realized that it was all. about. me.  (I realized this after emerging from a fabulous meeting with two smart and dedicated assessment colleagues who are working on many aspects of the assessment projects I had listed, and then later another assessment project meeting that was equally generative and engaging with a small group of faculty from an academic department).

How’s that for a blind spot?!? (Ugh – how embarrassing!)

What I totally failed to represent in that post was the importance of doing this work with other people — and how they are a key source of my learning, and are also likely learning as well, and of course, we learn together (as represented so nicely in THIS article). One person engaging in the work of assessment learns only so much; a group of people engaging in the work of assessment together, thinking and reflecting together, planning together can learn so much more.  Wisdom of Crowds and all that. Duh, right?

But seriously, this speaks to why it’s so important for an entire campus community to be involved in assessment work — assessing student learning is no easy task and sometimes we don’t want to see (or we can’t see) what’s there to see! Cathy Davidson’s recent post “Why You May Be Blind to a Good Idea” in the Harvard Business Review nicely addresses the value of working with others as well:

…since we all see selectively but we don’t all select the same things, we can leverage the different ways we slice and dice the world. The trick, though, is we can only do this by first accepting that we each have limits: Everything we see means we’re missing something else. It’s that simple. And impossible to see.

When doing the hard but important work of academic assessment, it’s good to able to see as much as possible, as clearly as possible. I, for one, need others to help me avoid the blind spots and biases, and I am grateful I have such talented and caring blind spot prevention teams in my world.

She Had A Pleasant Elevation

She’s moving out in all directions …

Like this Talking Heads song, this is how my summer has been – moving out in all directions. Though to be clear –  I am NOT taking LSD in a field next to a Yoo-hoo beverage factory in Baltimore, Maryland (thanks for this information Wikipedia), nor am I lying in any grass. I have been working with my colleagues on lots and lots of assessment projects, all simultaneously. And it’s fun and exciting and draining and cool. (And busy.)

Let me do a brief inventory:

  • Assessment of Learning in the Academic Library
  • Student Affairs Assessment
  • Academic Department Assessment Reports – 2010-2011
  • Academic Department Assessment Plans – 2011-2012
  • Preparing for rolling out Department Review- Chapter 2: Student Learning
  • Liberal Arts Core Revision (and supporting myriad assessment projects associated with the current LAC outcomes)
  • NWCCU Accreditation – Standard One
  • Hiring and welcoming our new Assessment Research Coordinator
  • Hiring and welcoming our new Service Program Coordinator
  • Teaching LRN 305
  • And, and, and … let me just say it’s been a busy summer.

And oh yeah, I almost forgot! I have been working on  my own learning in the Assessment Leadership Academy — all in context of these various projects and my day-to-day work.

I have to say that in moving in all of these directions, I am, in fact,  having a pleasant elevation. Wanna know why? Because when I am engaged in this work, I am learning. And why? Because assessment is about learning. (Not to be redundant – but have I said that before? Like HERE, HERE, HERE, and HERE?)

Megan Oakleaf, in a recent article about assessing value in academic libraries, said it too in regard to why assessment in libraries is important:

Value research means hard work: hard work conducting research, hard work reflecting on results, hard work fine-tuning existing services and resources, and hard work developing new ones. However, it is certain that not engaging in the value conversation puts academic libraries in  an untenable situation. It is also certain that investigating and demonstrating library value is the right thing to do. Why? Because as librarians explore the value of library services and resources they provide, they learn. When librarians learn, they proactively deliver top-notch services and resources where they’re needed—to students completing their academic work; to faculty preparing publications, grant proposals, or tenure packages; to administrators seeking decision-making evidence. And when librarians deliver excellent services and resources, they make a difference for their users—they are valuable.

This summer has been all about learning — student learning, my learning, my colleagues’ learning, my institution’s learning — and making a difference (or at least trying to). And learning and making a difference are valuable. And *that’s* been my pleasant elevation for this summer (which has got to be way better than taking LSD in a field next to a Yoo-hoo beverage factory any old day).

(Not that I would know.)

Yah, What They Said!

I can't find a way to credit this picture because I can't seem to find its original source! Research conundrum!

A few of my favorite selections from chapters in Field Guide to Academic Leadership (edited by Robert Diamond), one of our books for the Assessment Leadership Academy:

“The most appropriate solutions to the problems lie in shared commitment to and responsibility for good practice.” ~Michael Theall, Evaluation and Assessment – An Institutional Context

“The opportunity for critical reflection — a chance to put our strong academic values of systematic inquiry and questioning of assumptions to use — is lost in the desire to get the thing done.”   ~ John Wergin, Academic Program Review

“…change is a scholarly act (Ramaley, 2000) … Informed, respectful, thoughtful dialogue is the greatest learning tool of any organization today, and few of us know how to do it.” ~ Judith Ramaley, Moving Mountains – Institutional Culture and Transformational Change

Learning To Dream Again

Catherine Hooper, an Organizational Communications major at Marylhurst University, recently finished her 9-credit PLA Portfolio for the following topics:

  • COL 427 Great Meetings: Planning and Facilitating Difficult Group Discussion
  • CCM 321 Small Group Communication
  • COL 426 Team Building: Managing Work Groups

In her Final Portfolio Reflection essay, Catherine wrote:

The PLA program was especially valuable to me because it provided tools to help me find what I was interested in and did well at, and then matched that to a degree program. Education is valuable and I have always wanted to complete my bachelor’s degree; the PLA program created an environment where experience and education are combined to build a strong resume and a sound career path while working on my degree.

Through the PLA program I learned to dream again and think about my future in a more creative way. It offered me opportunities to remember where I had been and what I had done in life and how that has molded me into the person I am today. PLA then helped me take that awareness to develop an educational degree plan that truly excited me.

Congratulations Catherine — May you continue to dream!