Look beyond the numbers

Information visualization teaches us the intricacies of displaying data graphically, and how one must look beyond first impressions to see the "truth" behind the data (you can see a gallery of "best" and "worst" examples here).

To take a (presumably) fictitious example: a line chart showing how the cost of living has gone down at the same time that sales of the automotive Big Three has gone up. A politician representing Michigan might be able to get considerable mileage (no pun intended) out of a chart like that. Problem is, the data tends may be true in and of themselves, but by placing them together, the impression is given that they are interrelated, when in fact, it's very likely that they're not.

The lesson is: the data doesn't necessarily represent the picture it intends to portray.

Recently I came across a website for an elite boarding school in the Northeast. In the Admissions section, there is a page detailing grants and financial aid, with the following sentence:

"Intent on enrolling an economically diverse student body, (the school) offers
financial aid to students, based on financial need. In this school year, $2.8
million of financial aid was awarded to 28 percent of the student body. "

A very admirable goal, and with board-in tuition at just shy of $39,000/year, $2.8 million in grant aid is truly nothing to sneeze at. This chart is offered as an outline of their distribution method:

Income Level	Grants more than $30,000	Grants from $15,001 - $30,000	Grants up to $15,000
$0 - $ 25,000	9	0	0
$25,001 - $ 50,000	5	1	0
$50,001 - $ 75,000	13	6	1
$75,001 - $100,000	5	6	2
Over $100,000	9	22	7
Total	41	35	10

After looking at it for a while, I decided to do the math, using the maximum dollar amounts for each grant award type (i.e., for this example, "grants more than $30,000" equals the full in-board price of $38,800) and came up with the following:

Income Level	% of grant recipients	% of grant money	Grant $ awarded (in dollars)
$0 - $ 25,000	10%	12.5%	349k
$25,001 - $ 50,000	7%	8%	224k
$50,001 - $ 75,000	24%	25%	699k
$75,001 - $100,000	15%	14.5%	404k
Over $100,000	44%	40%	1114k
Total			2790k

Let's take it a bit further -- remember, according to the opening statement, 72% of the student body is not receiving any grant aid, and it's reasonable to assume that those students come from households earning more than $100,000/year. If almost half of the remainding 28% are from >$100k households that are receiving grant aid of some kind, that means that roughly 85% of the total student body falls into that income category.

So, upon closer examination of the data, those for whom "economic diversity" is a major concern might want to start asking some detailed questions.

Of course, there is equal danger in using this data to make a potentially false judgement of the school as a haven for the well-to-do -- after all, I'd venture to guess that many of those >$100k households are middle-class two-earner families, just trying to stay afloat while giving their children to an academically advantageous environment.

The point here is not to use data to make snap judgements, but rather to advocate for a healthy skepticism of data representations, so that more informed decisions can be made.

Label, baby!

A central tenet of user centered design (UCD) is that the people who create a product -- content experts, technical experts, marketing, management and other stakeholders -- are not actual users of the product. Each has biases (both positive and negative) that are brought to the table and are inherent in the input and feedback that they provide. Armed with credentials in the UCD/HCI/usability field, part of my job is to hammer that concept home as much as possible.

So it's particularly humbling when my own biases are put on display.

One of my current projects is a new version of an online learning management system (LMS) that allows instructors to offer exams, training session, or a combination of the two to their students. Once once of these modules has been created, it can be scheduled according to preferred parameters and made available to be taken by students in the class.

In the programs current version, the interface has two views -- Instructor and Student. On the student side, these modules labeled Assignments. On the instructor side, they are labeled Scheduled Items. Part of the challenge of the new program version was to come up with a single label, to be used in both views, that best encompasses all three types of modules.

"Common sense" -- always dangerous in this line of work -- told me that Assignments was the best choice (i.e., a task or duty created by one person for completion by another), but debate within development and stakeholder meetings was spirited. Ordinarily, this would be a perfect opportunity for a card sorting exercise -- instead, I was asked to sit in on several design feedback conference calls with customers and try to glean consensus, which was not productive. Assignments did not have strong support, nor did Scheduled Items. One participant lobbied hard for Activities, which made everyone on our end of the phone roll their eyes.

With no way of gauging accurate preference or consensus over the phone, I decided to create an online survey and solicit input from my non-professional contact list. After providing some basic context information, I posed the following question:

In your opinion, which one of the following labels best describes the three types of modules outlined above?

• Assignments
• Tasks
• Exercises
• Projects
• Activities
• Other (Please Specify):

40 invitations to take the survey were sent out, with 20 people responding. The results:

• Assignments - 2
• Tasks - 1
• Exercises - 2
• Projects - 6
• Activities - 7
• Other - 2

Needless to say, this was eye-opening on several fronts. Project stakeholders started gaining an appreciation for the actual science behind UCD, understanding that conversations taking place in a conference call does not make for a foundation for good design decision-making.

For me personally, it was a huge and much-needed reality check. I am expected to be the in-house expert on such things, but when push comes to shove, the biases I bring to design can be just as potentially damaging as those of a "non-expert." Only by clearly defining and identifying the needs, challenges and preferences of actual users can information be effectively used by stakeholders to guide critical information design decisions.

"Is it safe . . .?"

I usually don't dedicate much time attention to visual design, as it's not (nor has it ever been) my strongest suit, but I can't resist with this one . . .

A former colleague, David Cugnasca, is very heavy into pinhole photography and has been selected to participate in an international conference/photography show later this year.

Here's the poster for the event, which immediately brought to mind Laurence Olivier wielding a dentist's pick as a tool of torture in 1976's Marathon Man: