Thông tin tài liệu
User Experience Re-Mastered: Your Guide to Getting the Right Design
86
me what you are thinking as you are grouping the cards. If you go quiet, I will
prompt you for feedback.”
Whenever participants make a change to a card, we strongly encourage them to
tell us about it. It helps us to understand why they are making the change. In a
group session, it offers us the opportunity to discuss the change with the group.
We typically ask questions like
John just made a good point. He refers to a “travel reservation” as a “travel
booking.” Does anyone else call it that?
or
Jane noticed that “couples-only resorts” is missing. Does anyone else book
“couples-only resorts?”
If anyone nods in agreement, we ask him/her to discuss the issue. We then ask
all the participants who agree to make the same change to their card(s). Par-
ticipants may not think to make a change until it is brought to their attention,
otherwise they may believe they are the only ones who feel a certain way and
do not want to be “different.” Encouraging the discussion helps us to decide
whether an issue is pervasive or limited to only one individual.
Participants typically make terminology and defi nition changes while they are
reviewing the cards. They may also notice objects that do not belong and remove
them during the review process. Most often, adding missing cards
and deleting cards that do not belong are not done until
the sorting stage – as participants begin to organize the
information.
Labeling Groups
Once the sorting is complete, the participants
need to name each of the groups. Give the fol-
lowing instructions:
Now I would like for you to name each of your
groups. How would you describe the cards in
each of these piles? You can use a single word,
phrase, or sentence. Please write the name of
each group on one of the blank cards and place
it on top of the group. Once you have fi nished,
please staple each group together, or if it is too
large to staple, use a rubber band. Finally, place all of
your bound groups in the envelope provided.
DATA ANALYSIS AND INTERPRETATION
There are several ways to analyze the plethora of data you will collect in
a card sort exercise. We describe here how to analyze the data via pro-
grams designed specifically for card sort analysis as well as with statistical
TIP
We prefer to
staple the groups
together because we do not
want cards falling out. If your
cards get mixed with others, your
data will be ruined; so make sure
your groups are secured and that each
participant’s groups remain separate!
We mark each envelope with the
participant’s number and seal it until
it is time to analyze the data. This
prevents cards from being
confused between
participants.
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Card Sorting
CHAPTER 3
87
packages (e.g., SPSS, SAS, STATISTICA ™ ) and spreadsheets. We also show
how to analyze data that computer programs cannot handle. Finally, we
walk you through an example to demonstrate how to interpret the results of
your study.
When testing a small number of participants (four or less) and a limited num-
ber of cards, some evaluators simply “eyeball” the card groupings. This is not
precise and can quickly become unmanageable when the number of partici-
pants increases. Cluster analysis allows you to quantify the data by calculat-
ing the strength of the perceived relationships between pairs of cards, based
on the frequency with which members of each possible pair appear together.
In other words, how frequently did participants pair two cards together in the
same group? The results are usually presented in a tree diagram or dendrogram
(see Figs 3.4 and 3.5 for two examples). This presents the distance between
pairs of objects, with 0.00 being closest and 1.00 being the maximum distance.
A distance of 1.00 means that none of the participants paired the two particu-
lar cards together; whereas 0.00 means that every participant paired those two
cards together.
FIGURE 3.4
Dendrogram for our
travel Web site using
EZCalc.
Books
Links to travel gear sites
Luggage
Travel games
Family friendly travel information
Currency
Languages
Tipping information
Featured destinations
Travel alerts
Travel deals
Weekly travel polls
Chat with travel agents
Chat with travelers
Post and read questions on bulletin boards
Rate destinations
Read reviews
(Average)
0.50 1.00
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
88
Create a new message
Send current message
Attach file to a message
Spell-check current message
Reply to a message
Forward a message
Print a message
Get new messages
View next message
Delete a message
Save message to a file
Append message to a file
Create a new folder
Delete an existing folder
Rename an existing folder
View another folder
Overview of folders
Delete the trash folder
Move message between folders
Copy message between folders
Overview of messages in folder
0
2000
Complete linkageSingle linkage
4000
6000
8000
10000
12000
14000
16000
18000
20000
22000
24000
26000
28000
FIGURE 3.5
Tree diagram of
WebCAT data analysis
for an e-mail system.
BRIEF DESCRIPTION OF HOW PROGRAMS
CLUSTER ITEMS
Cluster analysis can be complex, but we can describe it only briefl y here. To learn more
about it, refer to Aldenderfer and Blashfi eld (1984), Lewis (1991), or Romesburg (1984).
The actual math behind cluster analysis can vary a bit, but the technique used in most
computer programs is called the “amalgamation” method. Clustering begins with every
item being its own single-item cluster. Let’s continue with our travel example. Below are
eight items from a card sort:
Participants sort the items into groups. Then every item’s difference score with every
other item is computed (i.e., considered pair-by-pair). Those with the closest (smallest)
difference scores are then joined. The more participants who paired two items together,
Hotel reservation Airplane ticket Rental auto Rental drop-off
point
Frequent-guest
credit
Frequent-fl yer
miles
Rental pick-up
point
Featured
destinations
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Card Sorting
CHAPTER 3
89
the shorter the distance. However, not all the items are necessarily paired at this step. It
is entirely possible (and in fact most probable) that some or many items will not be joined
with anything until a later “round” or more than two items may be joined. So after Round 1,
you may have the following:
Hotel reservation and frequent-guest credit ■
Airplane ticket and frequent-fl yer miles ■
Rental auto, pick-up point, and drop-off point ■
Featured destinations ■
Now that you have several groups comprised of items, the question is “How do you con-
tinue to join clusters?” There are several different amalgamation (or linkage) rules available
to decide how groups should next be clustered, and some programs allow you to choose
the rule used. Below is a description of three common rules.
Single Linkage
If any members of the groups are very similar (i.e., small distance score because many
participants have sorted them together), the groups will be joined. So if “frequent-guest
credit” and “frequent-fl yer miles” are extremely similar, it does not matter how different
“hotel reservation” is from “airplane ticket” (see Round 1 groupings above); they will be
grouped in Round 2.
This method is commonly called the “nearest neighbor” method, because it takes only two
near neighbors to join both groups. Single linkage is useful for producing long strings of
loosely related clusters. It focuses on the similarities among groups.
Complete Linkage
This is effectively the opposite of single linkage. Complete linkage considers the most
dissimilar pair of items when determining whether to join groups. Therefore, it doesn’t mat-
ter how extremely similar “frequent-guest credit” and “frequent-fl yer miles” are; if “hotel
reservation” and “airplane ticket” are extremely dissimilar (because few participants sorted
them together), they will not be joined into the same cluster at this stage (see “Round 1”
groupings above).
Not surprisingly, this method is commonly called the “furthest neighbor” method, because
the joining rule considers the difference score of the most dissimilar (i.e., largest difference)
pairs. Complete linkage is useful for producing very tightly related groups.
Average Linkage
This method attempts to balance the two methods above by taking the average of the
difference scores for all the pairs when deciding whether groups should be joined. So
the difference in score between “frequent-guest credit” and “frequent-fl yer miles” may
be low (very similar), and the difference score of “hotel reservation” and “airplane ticket”
may be high but, when averaged, the overall difference score will be somewhere in the
middle (see Round 1 groupings above). Now the program will look at the averaged score
to decide whether “hotel reservation” and “frequent-guest credit” should be joined with
“airplane ticket” and “frequent-fl yer miles” or whether the fi rst group is closer to the third
group, “rental auto” and “rental pick-up point.”
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
90
SUGGESTED RESOURCES FOR ADDITIONAL
READING
If you would like to learn more about cluster analysis, you can refer to:
Aldenderfer, M. S. & Blashfi eld, R. K. (1984). ■ Cluster analysis. Sage
University paper series on quantitative applications in the social sciences,
No. 07-044. Beverly Hills, (CA): Sage Publications.
Lewis, S. (1991). Cluster analysis as a technique to guide interface design. ■
Journal of Man-Machine Studies, 10 , 267–280.
Romesburg, C. H. (1984). ■ Cluster analysis for researchers. Belmont, (CA):
Lifetime Learning Publications (Wadsworth).
You can analyze the data from a card sort with a software program specifi cally
designed for card sorting or with any standard statistics package. We will describe
each of the programs available and why you would use it.
Analysis with a Card Sorting Program
At the time of publication, there are at least four programs available on ■
the Web that are designed specifi cally for analyzing card sort data: NIST’s
WebCAT® ( http://zing.ncsl.nist.gov/WebTools/WebCAT/overview.html )
WebSort ( http://www.websort.net/ ) ■
CardZort/CardCluster ( http://condor.depaul.edu/~jtoro/cardzort/ ■
cardzort.htm )
XSort ( http://www.xsortapp.com/ ) ■
UserZoom ( http://www.userzoom.com/online-card-sorting-study ) ■
OptimalSort ( http://www.optimalsort.com ) ■
Data analysis using these tools has been found to be quicker and easier than
using manual methods (Zavod, Rickert & Brown, 2002).
Analysis with a Statistics Package
Statistical packages like SAS, SPSS, and STATISTICA are not as easy to use
as specialized card sort programs when analyzing card sort data; but when
you have over 100 cards in a sort, some packages cannot be used. A program
like SPSS is necessary, but any package that has cluster analysis capabilities
will do.
Analysis with a Spreadsheet Package
Most card sort programs have a maximum number of cards that they can
support. If you have a very large set of cards, a spreadsheet (e.g., Microsoft
Excel) can be used for analysis. The discussion of how to accomplish this
is complex and beyond the scope of this book. You can fi nd an excellent,
step-by-step description of analyzing the data with a spreadsheet tool at
http://www. boxesandarrows.com/view/analyzing_card_sort_results_with_a_
spreadsheet_ template .
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Card Sorting
CHAPTER 3
91
Data That Computer Programs Cannot Handle
Computer programs can be great, but they often do not do all the analysis for
you. Below are some of the issues that we have encountered when using differ-
ent electronic programs. Although the data analysis for these elements is a little
awkward, we think the value that the data bring makes them worth collecting.
ADDING OR RENAMING OBJECTS
One of the basic requirements of cluster analysis is that all participants must
have the exact set of cards in terms of name and number. If participants renamed
any of the objects or if they added any cards, you will not be able to add this
information into the program. You will need to record this information for each
participant on a sheet of paper and analyze it separately. The number of cards
added or changed tends to be very small but it is an extra step to take. Returning
to our earlier example, you notice that Participant 1 added the object “airport
code.” Write this down and then tally the number of other participants who did
the same thing. At the end, you will likely have a small list of added and renamed
objects, along with the number of participants who made those changes. Based
on the number of participants who added it, you can assess its importance.
GROUP NAMES
The group names that participants provide are not presented in the analysis. You
will need to record the pile names that participants suggested and do your best
to match them to the results. We typically write down the names of each group
for each participant and look for similarities at the end. How many participants
created an “Airline Travel” group? How many created a “Hotel” group? When
examining the dendrogram, you will notice clusters of objects. See if there is
a match between those clusters and the names of the groups that participants
created.
DUPLICATE OBJECTS
As we discussed earlier, sometimes participants ask to place an item in multiple
locations. Because the computer programs available do not allow you to enter
the same card more than once and you must have the same number of cards for
each participant, include the original card in the group the participant placed
it. The duplicate cards placed in the secondary groups will have to be examined
and noted manually.
DELETED OBJECTS
EZCalc is the only program we are aware of that can handle discards automati-
cally, but IBM has pulled EZCalc off its main site. The only location for down-
loading EZCalc is http://www.tripledogs.com/ibm-usability/ . Many computer
programs cannot deal with deleted cards. For these programs, if you have allowed
participants to create a discard or miscellaneous pile of cards that they do not
believe belong in the sort, there is a workaround you need to do. You cannot
enter this collection of discarded cards as a group into a computer program since
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
92
the cluster analysis would treat these cards as a group of objects that participants
believe are related. In reality, these cards are not related to any of the other cards.
Place each rejected card in a group by itself to demonstrate that it is not related
to any other card in the cluster analysis. For example, if participants placed
“Frequent-Flyer Miles,” “Companions,” and “Meal Requests” in the discard pile,
you should enter “Frequent-Flyer Miles” in one group, “Companions” in a sec-
ond group, and “Meal Requests” in a third group.
Interpreting the Results
You now have a collection of rich data. The dendrogram displays groups of
objects that the majority of participants believe belong together.
Changes that participants make to cards can make interpretation of the results
tricky. When a deleted object is repeatedly placed in a group by itself (or left out,
in the case of EZCalc) , you may see it on a branch by itself or loosely attached
to a group that it really doesn’t belong with. Additionally, if participants place
an object in multiple groups, they may not have agreed on the “best” location
to place it. Consequently, you may fi nd the object is living on a branch by itself
or loosely attached to a group that it really doesn’t belong with. You must use
your knowledge of the domain or product to make adjustments when ambigu-
ity exists. Use the additional data you collected like new objects, group names,
changed terminology, and think-aloud data to help interpret the data.
Let’s walk through our travel example and interpret the results of our dendrogram
shown earlier in Fig. 3.4 . Using our domain knowledge and the group labels
participants provided in the card sort, we have named each of the clusters in
the dendrogram (see Fig. 3.6 ). We appear to have four clear groups: “Products,”
“Resources,” “News,” and “Opinions.”
It is important to note that the card sort methodology will not provide you with
information about the type of architecture you should use (e.g., tabs, menus).
This decision must be made by a design professional. Instead, the tree diagram
demonstrates how participants expect to fi nd information grouped. In the case
of a Web-based application with tabs, the tree may present the recommended
name of the tab and the elements that should be contained within that particu-
lar tab.
Now, you should examine the list of changes that participants made (e.g., renamed
cards, additional cards) to discover whether there is high agreement among
participants.
What objects did participants feel you were missing? ■
What objects did participants feel did not belong? ■
What are all the terminology changes participants made? ■
What defi nitions did participants change? ■
What items did users want in multiple locations? ■
Use this information to determine whether your product needs to add or
remove information or tasks to be useful to participants. You may recommend
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Card Sorting
CHAPTER 3
93
to the team that they conduct a competitive analysis (if they haven’t already) to
discover whether other products support such functionality. Similarly, use the
information about deleted objects to recommend the team to examine whether
specifi c information or tasks are unnecessary.
Terminology can be specifi c to a company, area of the country, or individual. With
each terminology change, you will need to investigate whether it is a “standard” –
and therefore needs to be incorporated – or whether there are several different
possible terms. When several terms exist, you will want to use the most common
term but allow your product to be customized so that it is clear to all your users.
Finally, examine the defi nition changes. Were the changes minor – simply an
issue of clarifi cation? If so, there isn’t anything to change in your product. If, how-
ever, there were many changes, you have an issue. This may mean that the prod-
uct development team does not have a good grasp of the domain or that there is
disagreement within the team about what certain features of the product do.
COMMUNICATE THE FINDINGS
Preparing to Communicate Your Findings
The specifi c data that you communicate to product teams can vary depending
upon the activity you conducted, but some elements of how you communicate
the results are the same regardless of the method.
FIGURE 3.6
Dendrogram of a
travel Web site card
sort with group names
added.
Books
Links to travel gear sites
Luggage
Travel games
Family friendly travel information
Currency
Languages
Tipping information
Featured destinations
Travel alerts
Travel deals
Weekly travel polls
Chat with travel agents
Chat with travelers
Post and read questions on bulletin boards
Rate destinations
Read reviews
(Average)
0.50 1.00
Products
Resources
News
Opinions
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
94
When we present the results of a card sort
analysis to executives or teams, we present the
actual dendrogram generated by the applica-
tion (as in Fig. 3.6 ) and a simple table to review
(see Fig. 3.7 ). We also present a table of changes
that participants made to the cards (added
objects, deleted objects, terminology changes,
and defi nition changes) and any sketches the
designers may have produced to illustrate the
recommendations.
As with all the other user requirement methodolo-
gies, the card sort is a valuable addition to your
software requirement documentation. These results
can be incorporated into documentation such as
the Detailed Design Document. Ideally, additional
user requirement techniques should be used along the way to capture new require-
ments and verify your current requirements.
MODIFICATIONS
Below are a few modifi cations on the card sorting technique we have presented.
You can limit the number of groups users can create, use computerized tools for
the sort instead of physical cards, provide the groups for users to place the cards
in, ask users to describe the items they would fi nd in a particular category, or
physically place groups that are related closer to each other.
Limit the Number of Groups
You may need to limit the number of groups a participant can create. For exam-
ple, if you are designing a Web site and your company has a standard of no
more than seven tabs, you can ask participants to create seven or fewer groups.
Alternatively, you can initially allow participants to group the cards as they see
fi t; then, if they create more than seven groups, ask them to regroup their cards
into higher-level groups. In the second case, you should staple all the lower-level
groups together and then bind the higher-level groups together with a rubber
band. This will allow you to see and analyze both levels of groupings.
Electronic Card Sorting
There are tools available that allow users to sort the cards electronically rather
than using physical cards (e.g., OptimalSort, WebSort, xSort, and CardZort). Elec-
tronic card sorting can save you time during the data analysis phase because
the sorts are automatically saved in the computer. Another advantage is that,
depending on the number of cards, users can see all the cards available for sort-
ing at the same time. Unless you have a very large work surface for users to
spread their physical cards on, this is not possible for manual card sorts. Elec-
tronic sorting has the disadvantage that, if you run a group session, you will
FIGURE 3.7
Travel card sort table
of recommendations.
Tab name
Resources
Languages
Currency
Family friendly travel information
News
Travel alerts
Featured destinations
Weekly travel polls
Opinions
Post and read questions on bulletin boards
Chat with travel agents
Rate destinations
Products
Luggage
Books
Links to travel gear sites
Objects to be located within the tab
Tipping information
Travel deals
Read reviews
Travel games
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Card Sorting
CHAPTER 3
95
need a separate computer for each participant. This means money and potential
technical issues. In addition, you need to provide a brief training session to
explain how to use the software. Even with training, the user interface may be
diffi cult for users to get the hang of.
Some tools support remote testing, which allows you to gather data from users
anywhere. However, users may have a more diffi cult time without a facilitator in
the room to answer questions.
Unfortunately, none of the computer-based programs provides a defi nition with
the objects. Also, they do not allow users to add, delete, or rename the objects.
In our opinion, this is a serious shortcoming of the tools and the reason why we
do not use them.
SUGGESTED RESOURCES FOR ADDITIONAL
READING
Prename the Groups
You may already know the buckets that the objects being sorted must fi t into.
Going back to our Web site example, if you cannot completely redesign your
site, you may want to provide participants with the names of each tab, section,
or page of your site. Provide participants with a “placemat” for each group. The
placemat should state the name of the group and provide a clear description of
it. Participants would then be tasked with determining what objects fi t into the
predetermined groups.
To go one step further, you may have the structure for your entire application
already laid out and simply want to fi nd out whether you are correct.
The article below provides a nice comparison of some of the automated card sorting tools
available (at the time of publication) if electronic card sorting is of interest to you:
Zavod, M. J., Rickert, D. E. & Brown, S. H. (2002). The automated card-sort as an ■
interface design tool: A comparison of products. In : Proceedings of the human
factors and ergonomics society 46th annual meeting, Baltimore, MD,
30 September–4 October, pp. 646–650.
EDITOR’S NOTE: CLOSED AND REVERSE CARD SORTING
The last example where you provide users with the names of categories and then put items
into those categories is called closed card sorting. Closed sorting is useful when you are
verifying an existing hierarchy or structure (e.g., the main menu of an application or Web
site) or adding new items to an existing structure. Closed sorting can be a follow-up to
open sorting and be used to validate the categories that emerged from the open sorting.
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
[...]... visible to all participants This is important for keeping the meeting focused Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 115 116 User Experience Re-Mastered: Your Guide to Getting the Right Design 4 Describe the brainstorming process to the group and explain the key principles and ground rules of your brainstorming session Write these principles and other ground rules on the. .. www.verypdf.com to remove this watermark 99 100 User Experience Re-Mastered: Your Guide to Getting the Right Design in another office The conference room table worked best; one participant especially liked the chair on wheels so he could roll up and down next to the table looking at his groupings Other participants sorted the cards standing up so they could reach along the table to work with the cards they had... Copyright © 2010 Elsevier, Inc All rights Reserved Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 108 User Experience Re-Mastered: Your Guide to Getting the Right Design 4 Discussing, critiquing, and possibly prioritizing the brainstorming results for subsequent action (this last step is often called the “convergent” phase where there is a winnowing of all the ideas into the. ..96 User Experience Re-Mastered: Your Guide to Getting the Right Design Reverse card sorting is similar to closed sorting In reverse card sorting, participants are asked to place cards that represent navigation items onto a diagram of a hierarchy (or other structure) and optionally, rate how certain they are that they are putting the card into the right place on the hierarchy The average... and listened to a lecture; the third group took a field trip to a toy store Each group then engaged in a brainstorming exercise According to Kelley, the group that actually went to the toy store generated more and better ideas than the other two groups If your brainstorming session dealt with how users could find a single item in a huge list, you could ask your participants to spend some time looking... decided to let them try the card sorting for themselves The danger, of course, was that they would remain in their own frame and not get beyond creating once again the site they knew Just a little prodding to “think about the users,” however, made these internal project team members realize for themselves both that they could put themselves into the users’ frame and that, once in that frame, they could... prioritize them later After the brainstorming session: a Designate a specific person or team to handle all the data after the session b Capture all the ideas and record which ideas will be considered further Catalog the ideas, preferably in a database so ideas that were not Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 117 118 User Experience Re-Mastered: Your Guide to Getting the. .. illnesses they knew and (2) what illness was treated by which remedy From these two questions, he was able to compare response Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 119 120 User Experience Re-Mastered: Your Guide to Getting the Right Design frequencies by gender, age, place of birth, and other factors The results of free listing can be used to rank order the words... the site, at that time, reflected the organization of the division and the research that it funds, the project team agreed that the mission of the Web site was to be the primary place that people come to for information on preventing cancer FIGURE 3.8 The Web site before card sorting Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 97 98 User Experience Re-Mastered: Your Guide. .. time to time and doing a debrief interview as each person finished Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Card Sorting CHAPTER 3 When the participants had sorted all the cards, we gave them the colored cards and asked them to name each of their groups We also asked them to place the groups on the table in the approximate configuration that they would expect to find the . PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
94
When we present the. purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
User Experience Re-Mastered: Your Guide to Getting the Right Design
100
in another offi
Ngày đăng: 26/01/2014, 14:20
Xem thêm: Tài liệu User Experience Re-Mastered Your Guide to Getting the Right Design- P3 pdf, Tài liệu User Experience Re-Mastered Your Guide to Getting the Right Design- P3 pdf, CHAPTER 1. What Is Usability? (Jakob Nielsen), CHAPTER 12. Inspections of the User Interface (Debbie Stone, Caroline Jarrett, Mark Woodroffe, and Shailey Minocha)