“A” is for actionable!

Assessment geeks like me refer to this as “closing the loop” — using our assessment findings (whether direct or indirect) to make improvements in our assignments, course design, instructional methodologies, programs, or our student services. Or, when appropriate, using our assessment findings to showcase what we are doing really well!

In the Assessment Program at my university, we constantly try to ask this question: What can we do with this data? We often all have the What? and we might even have the So What? But then …

NOW WHAT?

Simply put: How will what we learn here help us improve? What actions should we take accordingly?

So now we are writing about some of these assessment projects we’re working on making more actionable in my colleague’s Actionable Data blog. (By “we,” I mean “she.”)

(Oh – who is she? Her name is Kim Firth, and she’s an awesome data geek and a very cool colleague.)

Here’s part 1 of several posts to come about a project we are working on (again, by “we” I mean “she”): the revision of our Alumni Survey. Read it and weep. Ok, don’t weep – it’s not sad. If anything it should make you happy that we’re being so intentional about making this survey actionable.

Lessons from an attempt at action-focused survey revision

by Susan Marcus, co-chair of Marylhurst University’s Assessment Committee and faculty / curriculum specialist, Accelerated Online Programs

Do we know they know?
That’s why we do assessment
Don’t fear the “A” word

But how do we know?
Oh, the many ways we have
Bring in the “A” word

Are your needs unique?
Let’s uncover ways to help
Talk to your “A” team

(Have I mentioned how much I love my colleagues?)

This is a more recent picture of him with his new bike at the age of 4 and 30 seconds:

Some pretty major things needed to happen to get from picture #1 to picture #2. For example, in our family:

• Four year-olds need to be able to put on their own socks.
• Four year-olds need to make their beds when they get up in the morning.
• Four year-olds need to take their plates to the kitchen after they are done eating.
• Four year-olds, for goodness sake, need to feed themselves!!!

That’s right. We instituted milestones.

Milestones are constructed to provide reference points along the road. This can be used to reassure travelers that the proper path is being followed, and to indicate either distance traveled or the remaining distance to a destination. ~Wikipedia

Milestones, in this way, promote learning and help us assess how well we are doing toward a learning outcome. In the world of educational assessment, we call milestones a kind of “formative assessment.”

In my family, it’s what got us sanely from three to four.

Ok, ok, in all fairness, we really created milestones so to preserve what’s left of our parental sanity, but we also wanted the kid to have a goal (being four), to work toward something significant and important (and I quote, “Mama, I fed myself my dinner all by myself; now I can be four!”), and to have a sense of accomplishment when he got there.

The payoff for reaching these milestones? Well, not only did the kid actually turn four (which frankly seemed significant enough to all of us), but he also got that new bike in picture #2.

And guess what! To prove to us that he had in fact turned four, he ate 2 pieces of this birthday cake!

All by himself!

• Team Building: Managing Work Groups
• Great Meetings! Planning and Facilitating Difficult Group Discussion
• Leadership Communication
• Managing Transitions
• Conflict Management
• Negotiation
• Legal Issues of Managing Human Organization
• HR for the Career Minded Professional
• Creating Inclusive Trust Cultures
• Listening Skills
• Public Presentations

Congratulations, Bobbi!

In Bobbi’s Final Reflection Essay she wrote, in part,

My biggest ‘ah-hah’ moment was towards the end of LRN 312 when I was seriously getting tired of writing essays. My inner voice was telling me that I was getting tired and I wanted to learn and write about something that I did not already know. Then it occurred to me that the PLA process and writing itself was, in fact, teaching me something new. I was learning how to and becoming a masterful writer!

Bobbi talks about her PLA journey and simple tips for success here:

• Effective Listening
• Small Group Communication
• Conflict Management
• Nonverbal Communication
• Planning & Facilitating Difficult Group Discussions
• Communication of Self-Esteem
• Team Building
• Intercultural Communication
• Interpersonal Communication
• Poverty and Schooling

In his Final Reflection Essay, Greg wrote, in part, “I learned how to analyze and evaluate theoretical texts from scholarly sources and to apply this material to my own life experience.  Organizing my essays produced a continuum, and with the completion of each paper the method became smoother.”  Greg talks more about his experiences with PLA and tips for PLA students here:

As part of her undergraduate program, Velda earned 25 credits for the following courses through Prior Learning Assessment:

• Effective Listening
• Public Presentations
• Human Emotion
• Publications and Performance
• Women Working for Change
• Readings and Special Project in the Teachings of Abd’ul-Bahá

In her Final Reflection Essay, Velda talked about how writing for PLA credits increased her confidence in her writing skills, so that now a 6-8 page term paper seems like an easy task after working on a 20-25 page PLA essay. She also commented that, “My love of writing has been increased as well as my skill in the art, and I look forward to my next terms with excitement.”

In this video Velda shares her experience in the PLA program – her enthusiasm for life-long learning is infectious!

• Introduction to Criminal Justice
• Introduction to Law – Fundamentals
• Introduction to Law – Substantive Areas
• Criminal Law
• Litigation
• Evidence
• Observation and Guidance I
• Environment and Curriculum II
• Instructional Strategies – Reading
• Interpersonal Communication
• Effective Listening
• Introduction to Christian Bible
• Nutrition
• Backpacking: Map and Compass

CONGRATULATIONS, CARLEEN!

One thing of note that Carleen included in her Final Reflection Essay was this:

As I was writing each essay, I was totally caught up in what I was writing about, and became aware how much knowledge I gained through different experiences.  Some of these experiences were over 10 years ago, but I was back at that time as I was writing the essay, and reading material showed me how much I know about a certain subject.

Hear Carleen talk more about her PLA experience here:

Spirituality is the eternal human yearning to be connected with something larger that one’s own ego.

BUT

This “yearning” can connect us with great evil, as well as great good.

SO

Clearly, higher education ought to be inviting students to give thoughtful consideration to the energy fields of both sorts on their way to a diploma.

First Official School Photo

The report card is not a card that says “A” or “C” or any other grade, but is, in fact, a rubric.  There is a description of where his learning levels are in several categories, accordingly:

(E) = Exceeding – consistently exceeding grade-level expectations; a strength

(M) = Meeting – developmentally appropriate or meeting grade-level expectations

(D) = Developing – working towards grade-level expectations

(X) – Not assessed at this time; not applicable

For each category, there are specific learning items that are assessed using this framework (some are skills; some are knowledge areas; and a lot is behavior, as you might expect for students who are 3 and 11/12ths years old). Here is an example from the listening category:

LISTENING

• Effort – M
• Demonstrates comprehension in the daily routine – M
• Listens attentively to spoken language – M

(No comment here about how I would assess his listening skills. Let me just say we might have a case of grade inflation happening here. Or an inability to transfer skills from one context to another. Either way … )

Thus, across several categories and skills, we now know where his teacher sees his strengths and where we can help support his improvement. For example, he can count from 1-6 (E) and sort objects by color, shape, and size (E), but he needs more work in demonstrating self-control (D) and accepting responsibility for his own actions (D). He is doing as expected in recognizing his own name in print (M) and cutting across paper with scissors (M).

This takes us to the parent-teacher conference, which was also about assessing his learning and was evidence-based. His teacher had an iPad with about 30 pictures of Mac taken from the beginning of the year. Together, we looked at evidence of how he held a marker in September, October, and November; we were able to see differences in technique by looking at actual letters, shapes, and pictures he had drawn in an accompanying portfolio of work. She also had samples of his writing in which we saw evidence of how he wrote M – A – C in September compared to how he writes M – A – C now (not much improvement there, frankly. The M is still upside down thus spelling WAC instead of MAC. No comment…)

My point?

THIS IS ASSESSMENT, AND IT’S AWESOME!

We pose questions: What and how well is he learning? What evidence do we have? And what do we need to keep working on? And in answering these questions, we learn and his teacher learns and his school learns! Assessment = Learning = Assessment = Learning and around and around we go through the learning cycle. And we love it because we care.

Overall, Mac seems to be learning and doing pretty well in school (which is awesome considering that he is 3 and 11/12ths years old and has trouble listening … but apparently only with us). To provide further evidence that he is learning, here is a conversation between Mac and his dad this morning:

Mac: I want to wear these pants today.

Dado: Cool! These are cords!

Mac: Oh – I can’t touch those. I am not supposed to play with cords. They’re dangerous!

This after yesterday’s moment of inquiry, accordingly:

Mac: After the champion wins, is that when they get the chips?

Dado: The chips? What chips?

Mac: Yeah, the champion chips, Dado.

(Yep – most certainly grade inflation.)

FIRST: WHAT DO YOU WANT TO LEARN?

The first step, of course, is to identify what you want to learn. What would the learning outcomes of a visit be? Surely you will learn stuff you didn’t know you wanted to learn too, but if you can identify a focus for yourself and your institution, proposing a visit and the visit itself will be a lot more focused.

For example, for my recent visit to a university in SoCal, I identified these focus areas and shared them with my colleague there:

• What you are doing with Liberal Arts assessment, specifically using the VALUE rubrics and your institution’s core revision process?
• What the librarians are doing with their assessment work?  (You’d mentioned that they were using SAILS – I’m wondering if it would be possible to meet with them to learn more about how they are doing assessment?)
• A sense of how the Assessment Committee(s) work — their structure, charge, participants, etc.
• More about your Program Review processes and outcomes
• Your role in building the culture of evidence / assessment

This list helped her know who to set up meetings with and what materials I might be interested in seeing. (Indeed, it was a rather large list, but in all fairness, I was coming from out of town and only had one day to meet with them. I was trying to be comprehensive.)

TYPES OF VISITS

In my experience, there are two main types of learning visits:

1) Problem-based: This kind of visit is intended to help you work on a problem you or your institution has. For example, I arranged a phone visit for myself and 3 colleagues from my university to talk with a person at another institution in Chicago about how they manage the assessment of student learning in an outcomes-based liberal arts curriculum; this was a very real problem for us at the time. (Turns out it was for them too!)

2) Topic-based:  This kind of visit is more focused on a specific shared topic, such as “general education learning outcomes” or “facilitating internships.”  A while back, I set up a session with a person using the Mahara ePortfolio system at a different institution because we wanted to explore this system and see it in action. We used a web-based desktop sharing system and the phone, and she kindly took us on a 45-minute tour of how they are using Mahara ePortfolios. Nice!

IDENTIFYING WHERE & WITH WHOM TO VISIT

Figuring out where and with whom to visit per your desired learning outcomes is likely the next step. Here are some ideas for how to do so:

1) Look to your local network. Are there colleagues with similar job responsibilities at institutions near you? Do you know anyone at an institution near you that can connect you? Do any of your current colleagues have connections at these institutions that they could leverage for you?

2) Look to your distant network. Have you connected with folks at conferences that you can reconnect with? Even if you can’t visit physically, with the phone, Skype, or with other technologies you can visit virtually. I have a “coffee date” about once a term with a colleague from another institution across the country who I met at a conference; we both get a cup of coffee and talk on the phone for about an hour, and just learn from each other.

3) Look to your virtual network. If you use LinkedIn or Twitter, search for and follow people who are in similar roles or who have identified projects they are working on that are similar to yours. This is how I connected with the Mahara ePortfolio person; I had learned a lot from her by just following her on Twitter, and then when my colleagues and I were ready, I sent her a message with a few questions. From this initial conversation came the idea to have a short virtual meeting in which she took us on a tour.

PROPOSING THE VISIT

First, make contact and make a simple initial proposal. It might look something like this:

Hi there – My colleague XX shared with me your contact information because I am interested in learning about what you are doing with blah blah blah at your university; we are trying to implement this at my university as well, and I wonder if we might be able to set up a time to chat briefly about what is working and what’s not. Maybe we can learn some strategies from each other. Etc etc etc …

As demonstrated here, a proposal for a visit might be more compelling if you identify what you can bring to the table. What will be in the visit for them? What might you be able to contribute to the conversation?

Also, start small. You don’t need to visit for a whole day. Maybe you just begin with a short phone conversation, or maybe (if you’re close) you meet for lunch.

Finally, I think it’s useful to learn with others, as a team. If you can, take a colleague or two from your institution with you, or invite them to the phone / Skype conversation. Propose the person you’re meeting with do the same. The more the merrier!

DID YOU LEARN?

After the visit, assess your learning. Did you achieve your learning outcomes? If not, what might be next steps? What additional questions were raised for you, or what other resources should you explore? Likewise, was the person / institution you met with interested in learning more and continuing the conversation? If so, then maybe you all set up another learning visit with each other.

ONE LAST TIP

In the spirit of academic integrity,1) don’t take and just start using what is not yours and 2) give credit where credit is due. We all adopt and adapt ideas and materials all the time in higher education; if you want to borrow something from someone you’ve visited, ask permission and then attribute it. There may be nothing more irksome than being visited and sharing a rubric, for example, and not knowing the visitor adopted it or re-purposed it.

Happy visiting – learn lots!