Sunday, August 31, 2014

Grading Rubric, Why Oh Why Art Thou as Thou Art?

Having to construct a grading rubric has a way of bringing one down to earth with an unforgiving slam. You know, you are all caught up in visionary ideas of how to do your part in engaging student minds in various exciting and unquestionably marvelous ideas - oh the places we will go. We will read this, talk about that, pull apart and put back together that, create construct and write write write. Stretch and ponder and percolate and ruminate.

You see, I'm going to be back in the classroom this Fall (er, next week) after a few years doing other things. I'm still doing other things, but in addition, I'm about to land myself in front of two sets of computer science undergraduates. I'm psyched. Yes, I am. There is something that just says "yes" about teaching. So, along with a zillion other faculty around the western hemisphere I have spent much of the past several weeks whipping together "the plans". Two courses, one of which is called "Computer Science Education Research".

The chickens came home to roost first and foremost with that class. For CS Ed. Research I have teamed up to co-teach with a friend and colleague; together we have been putting together the pieces on a subject we both hold near and dear. How to conduct this type of research, how to write it up afterwards, how to present it to an audience.

I must say, my colleague writes a darned good grading rubric. She's great at it.  She reminds me that we must be concise and clear. State expectations. Avoid fuzziness. I'm good at fuzziness sometimes.

Back when I was in grad school and simultaneously teaching a course called Technical Writing for Computer Science Majors, I took advantage of workshops funded by a large grant to help instructors teach "substantial writing component" courses. Clearly that was my class. We weren't writing boring user manuals (much); we focused heavily on how to deconstruct a research paper. After we  had spent a fair amount of time talking about what a well constructed conference paper looked like (in CS Ed. of course), I would do one of my favorite activities which was to hand them a paper to perform a constructive critique on.

Predictably, they would tear it to shreds, because it wasn't a particularly well written paper. It needed a lot of help. Also predictably, and this happened semester after semester for 6 years, the students were stronger on the "criticism" than on the "constructive" part. They were often less than kind in their assessment of the unknown writer. This, in spite of our having discussed the role of a reviewer and the importance of remembering there was a person behind this paper. Someone who had worked very hard and done their very best.

There was always a deathly sick silence when I eventually told them I wrote the paper; it was my first ever attempt at a conference submission. It was, quite understandably, rejected. But rejected kindly, which was fortunate for my then fragile sense of researcher self. By the time I was in the front of the classroom several years later, I had developed a thick enough skin to take the worst kind of reviewer commentary (and if you have submitted enough papers you know what I'm talking about).

The point was made better than when I ever just said it - be kind when you must critique someone; there really are real people behind the written word; finally, you never know how things will come back around. My students were luckier than they might have been "in the real world" because I didn't hold anything they had said about the paper (or me, unwittingly) against them.

But back to that grading rubric. The substantial writing component workshops I took advocated for an idea that sounded really great at the time. The idea, roughly, was this: you get highly explicit about what constitutes an F, D, C, B. And what you indicate in the criteria for a B is that you get a B if you do everything you were supposed to do and you do it really well.

So how do you get an A? You go above and beyond. You do something that blows it out of the park. You get creative, innovative, insightful, or visionary. You do everything you were supposed to do for that B and then you come up with something I the instructor couldn't predict. I can't tell you in the rubric what that is, because then it wouldn't be creative would it?

And herein lay the problem. It sounds great doesn't it? Find a way to set that bar high and provide incentive for people to really GO FOR IT. The problem is....

Well, there are multiple problems. It's subjective. Boy is it subjective. My idea of mind blowingly innovative may not include your idea of mind blowingly innovative. My idea of creative may well not be yours. So right off the bat there is all sorts of room for bias. Not only that, students hate it because it is confusing and stressful and not at all the way they have likely ever been graded before. And notice I say "graded" and not "assessed" because let's be real here, it's a table with itemized instructions for obtaining a grade!

The people who ran the substantial writing component workshops and who advocated for this type of rubric claimed there was all sorts of literature backing up the effectiveness of this approach to grading writing intensive courses. They probably had that literature available, and knowing me I probably read it.

But it didn't work for me and I tried it in several different courses. I really worked at it to make it work. I was all for the idea  - sounds great for a free ranging thinker who likes to push the envelope and encourage the same in others. But not so great for people who see the world differently. And that's a lot of people. And in educational system that doesn't operate this way as often as it might. But frankly, the potential for rampant unrecognized bias is what really bothers me now. There are other ways to encourage creativity and innovation in class. I feel confident in saying we will be doing so.

As I worked with my colleague on our CS Ed. Research class I got tasked with making the first pass at several of our grading rubrics. Yuck. Ok, I said it: yuck. I'd much rather have worked on something else. But it has been very good for me. It reminds me that vision and grand ideas have to come down to earth sometimes, especially if you are in front of the classroom. At least for a while.  Having to push against my natural instinct to avoid the precision of the grading rubric, and having my colleague point out every time I have gone fuzzy, is an excellent opportunity for me to learn too.

Wednesday, August 6, 2014

A Newbie's Introduction to UX Speakeasy

Someone reminded me this evening that we created UX Speakeasy almost 3 years ago. I didn't realize so much time had gone by. We started with a dozen people and suddenly we have more people than we can fit in a good sized auditorium. Not that we ever meet in auditoriums. This month we met at Societe Brewing Company, hidden back off a busy street. Very pleasant atmosphere, lots of wood, and a very nice bartender.

As luck would have it, my friend Reynaldo was hanging out with me this week and expressed curiosity about UX Speakeasy. I asked him if he'd like to come along and give his perspective on things. He was all for it so I snuck him in. Board membership has its privileges. Reynaldo is on the small side and was feeling a bit shy
so he clung pretty close to my shoulder all evening. I didn't mind, because as long as he stuck with me it was easier to show him around and explain this eclectic group of people. Even better, I was able to hear what his experience was like as it happened.

Prior to the official Meetup we held a short Board meeting. Reynaldo was impressed by the fact that this group of people, meeting in a local brewery, were actually incorporated as a non-profit and concerned about making sure there was a structure, designated roles, goals and Vision. He was curious about what UX meant, and I whispered to him that this was sometimes a contentious question. I told him he'd be ok this evening as long as he didn't randomly swap out "UX" for "UI", or worse yet, "web design" in casual conversation . He understood, pointing out that sometimes people confused him with an ocean crab. (This would make sense if you met Reynaldo in person)

Out in the parking lot we had custom oven fired pizza by Red Oven Artisanal Pizza. One of the reasons Reynaldo and I hit it off when we first met is that I'm a vegetarian. So there was no chance I was going to get
us a seafood pizza. We both agreed the pizza was outstanding; it was custom made and fired in the stove as we watched. Somehow they managed to bring in an entire pizza making operation and kiln-like stove just for the evening. Reynaldo tried to pinch more pizza than he was entitled to.

Once the Meetup officially began I walked around saying hi to people and introducing Reynaldo. The guy's a bit of an introvert and didn't say much. But he wore a silly grin and enjoyed watching the 70+ happy attendees playing the card game UX Against Humanity.*

UX vocabulary can be confusing if you are new to the scene. So are the rules to this card game. I was having trouble trying to explain, mostly because I was taking the detailed instruction sheet too literally. It was Reynaldo who pointed out that the small groups of people hunched laughing around the barrel tables were ignoring most of the rules. At that point I decided he was doing just fine, so I begged off to wander around and take photographs.

When I got back from my photo shoot, Reynaldo explained his new understanding of the game. I share his
explanation with you in case you weren't there:

It's reminiscent of Mad Libs, if you are old enough to remember that game. There is a sentence with blanks you have to fill in and the funniest one wins each round. But in this case, the sentences were UX based and the fill in answers had to be chosen from phrases on cards in your hand - UX based as well. So you might end up with totally meaningless sentences like this:

"The future of UX depends upon a hipster's beard dandruff" 


"I'm no designer but have you thought about adding a double rainbow?"

Reynaldo thought this one was particularly funny:

"The worst UX superpower is sniffing your research participants" 

I should explain that Reynaldo comes from a sheltered background and tends to retreat into his shell when things get dicey.** So he felt a bit crabby with some phrases people created that used four letter words and anatomical references. Out of respect for his sensitivities I won't repeat them here.

More to his liking were these phrases:

"The future of UX hinges on one thing: a hair on the rim of your Starbucks cup"

"When I am wire framing I first like to sketch out relieving the cognitive load"

"When I am wire framing I first like to sketch out James Tiberius Kirk" 

"When I am wire framing I first like to sketch out cat memes"

It was a long evening; things got quite silly. Reynaldo and I had a good time, as did a lot of other attendees, judging by all the laughter and enthusiastic game play. He asked me how the group is going to top this event next month but I had to demur.

Instead, I left him to ponder:

"Life imitates: ________________"

Hint: Starbucks is not part of the answer.

*I have Reynaldo's approval to write about him. He read my post, and had the opportunity to edit it.

**His Latin-sounding name not withstanding, Reynaldo spent his formative years in China.