December 30th, 2008 by David Poteet

Doing a Carewords Survey for Imperial College London

Filed under Higher Education, Case Studies

We're working on a user experience project right now for Imperial College London's Research Support group. This group serves the needs of academics and administrative staff on the Imperial College campus as they pursue funding for research and manage ongoing research contracts. As part of the project we are conducting a Carewords Survey, which is an approach developed by Gerry McGovern for identifying the top tasks of your site visitors. You can see the actual survey we set up for Imperial College London below.

Gerry wrote recently about how top task performance heavily influences branding:

The essence of the brand is the experience customers have with a product or service. There is of course a visual component in branding but it has been vastly overemphasized. When we think of Google do we think of a colorful logo or a fast way to find stuff? Web brands, in particular, are functional. The successful ones help us do useful things.

His point jives with research conducted by User Interface Engineering on brand engagement before and after the use of a web site. UIE has shown that customer's connection to a particular brand is strongly affected by whether they are able to quickly and easily accomplish their goals on the web site.

While the main Imperial College website underwent a redesign in 2007, the structure and content of the Research Support site didn't see any major change – only an update to the template. It had a refresh to the "branding" as we typically think of it, but the actual customer experience wasn't any better. Here's a screenshot of the site as it exists as of this writing:

Imperial College London's Research Support web site as of Dec. 30, 2008

Imperial College London Research Support - BEFORE

Staff at Imperial have been telling the Research Support group for a while that the site is difficult to use. Most of them just pick up the phone rather than trying to sort through the site. Research Support recognized the problem but needed a clear plan of action.

This Fall we worked with Pamela Agar, Imperial's Director of Digital Media and Marketing, to develop a plan that looks something like this:

Analyze current site traffic and search engine data.
Interview 6-8 staff about their interactions with Research Support and use of the web site. We're basically following the same interview format we use for creating mental models, although we may not create full models from them. Pamela conducted these interviews, which were all recorded and transcribed so that we could analyze them.
From the interviews, create user-test scenarios that represent the top goals of most staff.
Conduct baseline usability tests with the current site.
Develop a list of "Customer Carewords" from the interviews, the current RS web site, search logs and other similar sites at research institutions.
Conduct a Carewords Survey with a large sample of staff at Imperial College.
On-Site: Conduct KJ brainstorming sessions with Research Support staff, and correlate this with what we learn from the interviews and Carewords Survey.
On-Site: Work collaboratively with Imperial staff to develop a paper prototype of the new site including the overall site architecture and layout of key pages.
On-Site: Conduct usability tests with these paper prototypes, with Research Support staff observing.
Revise wireframes based on the usability tests and create higher fidelity wireframes.
Re-test if necessary.
Develop a plan for creating, revising and migrating content from the current site to the new site.
Implement the wireframes within Imperial's CMS.

There's no one right way to do something like this. We chose these steps because we want to make sure that when we get to the design stage at Step 8, we can make informed decisions with a minimum of disagreement, and that we'll have a chance to test our designs and improve them.

What we've learned so far

We've already learned quite a lot from the user interviews and baseline testing. It's great to have someone like Pamela on the client side who can handle some of these steps for us. Here are some highlights of what we've learned:

People love the staff of Research Support. Everyone praised the team there for being helpful and knowledgeable.
Many of the questions people have about a funding proposal are very specific to that proposal and would be hard to answer in a general format on the web site. Most staff don't even expect that the web site would be able to answer their questions.
Because of this, there is probably a significant amount of content on the site that could be simplified or eliminated altogether.
The current web site failed to provide a successful path for almost every scenario we tested in baseline user testing. For example, users did not expect that they would find help with a funding proposal under "Grants Contracts and Agreements."

Carewords in action

A Carewords Survey helps you uncover the top tasks for your users, as well as the best words or phrases to lead them to those tasks. Building your site with these top tasks in mind raises brand engagement. In the case of Research Support, we're hoping that this will help us decide what to use for our main navigation, and what information and links to feature on the home page and key landing pages.

Compared to other research techniques, it's fairly inexpensive and easy to do. The simple version of the process is:

Build a list of about 100 words or phrases that people might be interested in on your web site. You can find the words by brainstorming, scouring your site and your competitors' sites, interviewing customers, looking at your search engine logs etc. Redundant words are fine – you're looking for the best wording.
Have 100 or more people choose their top 5-10 words from the list. After they've chosen them, have them sort them in order from most important to least important.
Score the words by giving 10 points to their top choice, 9 to their second, 8 to their third and so on.
You'll end up with a ranked list of words and phrases that very accurately reflect the priorities and top tasks of your audiences.

We created a web-based survey tool for doing a Carewords Survey. You can see a demo of the survey we created for Imperial College here: http://survey.newcitymedia.com/step_1.php?survey_id=ncm

This isn't the real version so you can't mess up our data. Go ahead and try it!

I plan to share the results of the survey after we run it next week. We'll also post updates on our on-site planning sessions.

Read More

Gerry McGovern writes a weekly email newsletter called "New Thinking" about managing customer-centric web sites. Read the latest issue about how top task performance heavily influences branding. You can also read more about Customer Carewords on his site.

Jared Spool of User Interface Engineering is going to be giving a virtual seminar called "The Road to Informed Decisions" on January 15, 2009. Watch the archive if you miss the live event.

What do you think?

What questions or ideas do you have about how we're approaching the online experience for Imperial College Research Support? Is there anything else you'd like to see from this case study?

Speak up.

Respect.NewCity will never distribute, sell or otherwise treat your information like its ours to run around all willy-nilly, hither and yon with. That's because we appreciate your contribution to the conversation.