Card Sorting: Designing user informed categories

Card sorting is a technique for informing Information Architecture. It allows the architect to collect data on users categorisation preferences and gives them an opportunity to observe users discussing their reasons behind their choices.

Over at Boxes and Arrows Donna Spencer and Todd Warfel describe it as:

A great, reliable, inexpensive method for finding patterns in how users would expect to find content or functionality. [1]

An example

Imagine you’re standing in the supermarket and you’ve been given a list of items to purchase. The list is in no particular order and is completely uncategorised. You read down the list:

  • Shopping List:
  • Chicken
  • Oranges
  • Sweetcorn
  • Cotton wool balls

In your head you’re already starting to think “Is the Chicken in the meat aisle or the freezer section”, “Do I find the Cotton wool balls with the baby stuff or with the cosmetics”

This is what a user is doing when they come to your site for the first time. They’re trying to find something and they’re looking at the sections on your navigation and trying to work out where to go first. Which section titles best describe what they are looking for.

So how does Card Sorting help us with this? Well we do something similar to standing our users in a supermarket with a shopping list, except this time we ask the to group the items and name the groups in a way that makes sense to them.

Note: The following method is for an observed/moderated experiment. There are a number of great tools available on the web to help you setup remote/unmoderated experiments, however, I find that while these are great for producing a lot of data quickly, you lose the context provided by observing the users as they discuss cards and move them between categories.

Before you start

Picking your cards

I work on the basis that 75 cards will take your users about an hour to sort. So consider how many cards your users will be able to sort effectively in the time available. Also consider which pages to sort. You should strip out any category pages, as these will be replaced by your users newly named groupings. Typically you should be looking at 2nd/3rd level pages.

Preparing your cards

There are 3 things that I’d recommend each of your cards having:

1. A unique ID
This is purely for administrative purposes. It allows you to identify the cards during the analysis phase. Make sure it’s random and doesn’t relate to the section that the page currently lives in (if you’re experimenting with an existing site). Make it small, you don’t want to lead to the users into groupings based on the ID.
2. A page title
If you’re testing an existing site you may be able to pull these straight from the page titles in the metadata. However, if your page titles have been optimised for SEO (or spammed with keywords) you may want to clean these up so that you don’t mislead the user.
Watch out for any pages that may have duplicate names like “FAQs”. Some sites have one of these pages per product or per section, consider renaming it to “Product Name – FAQs”.
3. A short description
Again this may be able to be pulled from your pages metadata. The same consideration applies to SEO optimised descriptions. If the page title is a little ambiguous use the description to supply some context to the page.

Here is an example based on our supermarket scenario:

Free range corn-fed British chickenThis delicious and richly flavoured bird has a characteristically golden colour to the flesh, which comes from its maize based diet. The hens are reared in the same way as our other free range birds. 

#1

If you’re any good with a spreadsheet I’d recommend setting up a mail merge to print these. Depending on who you’re asking to take part, their location and the time between the sorts talking place and being able to start the analysis, you may need several sets of cards printing.

Recruiting users

Consider who you’re going to recruit to represent your users. Make sure that your sample is as representative as possible of gender, age, nationality, skill and role/user type.

Card sorts can be done with an individual user but you tend to not get a lot of discussion with users sharing their thought process or opinions. For this reason I’d suggest groups of 2 or 3 users so that there is plenty of discussion, you’ll be able to hear where there is consensus and where there is disagreement.

When recruiting users I would recommend a minimum of 5 groups, as this will give you 5 unique sets of data to analysis. While this is just my opinion it is loosely based on Jakob Nielsen’s Why You Only Need to Test with 5 Users.

If your site or application has a diverse range of users then perhaps consider 3 groups of each user type, or persona if you have created these.

Grab a copy of the analysis spreadsheet

Donna Spencer, whose definition of card sorting I used to open this post, has done a lot of work in the area of card sorting. This includes the production of some fantastic resources and materials to help you with your card sorting.

While the post is 5 years old the resources are regularly updated to ensure that formulas continue to work in the latest versions of excel etc. So for that reason I’d like to point you directly to Donna’s post on the Rosenfeld website.

Running the experiment

After the planning, the preparation of the cards, recruiting your users and scheduling the sorts to take place – running the actual sorts is relatively straight forward. However, there are some things that you can do to make the sorts go more smoothly:

Have a facilitator

If you’ve run any workshops in the past you’ll be more than use to facilitation. You should consider using the same skills to facilitate an experiment. The sorts of things that you should lookout for are as follows:

  • That everyone in the group is having an equal input
  • That the group is working as a team and not working against each other
  • That the group remains on topic

Do your best not to lead a group into making decisions, but you can advise or clarify to some extent if the group are running into real difficulty.

Run through an example

Just as I have given the example of a supermarket in this blog post you should make up 10/15 cards in advance that you can show the group. I like to use items from around the house. This quickly shows that items can be categorised by room (bedroom, lounge, kitchen etc) or by type (electrical, furniture, soft furnishings etc).

Be sure to explain that both are valid ways of categorisation and that the users should go with the approach that they are most comfortable with.

Observing and scribing

As well as having a facilitator in the experiment I would suggest having someone available to observe and scribe. If you have access to an observation lab the scribe could sit in the observation suite taking notes and recording the discussion and actions made by the group. It is not essential that your scribe be hidden away, I have been known to sit in on the experiment, recording, note taking and in some case photographing the groups activities.

If you are unable to get a scribe it as at least worth considering recording the session. Most iPods/iPhones will record through the headphone socket using the headphones straight to mp3 and you’ll be grateful of having something to go back to when you’re writing up your findings.

Spare cards and envelopes

Although this is a relatively low-cost experiment you’ll need to raid the stationary cupboard a few times…

It’s always worthwhile having some blank cards with you. Some users may want to split content or duplicate it into multiple categories, like the cotton wool balls that live in both ‘baby stuff’ and ‘cosmetics’. Allowing your users to duplicate the card by hand and putting it in both categories will act as an aid when it comes to the analysis.

You should also have a number of large envelopes with you. I like to take large A4 envelopes with me and allow the group to write on them and use them as placeholders for their categories. At the end of the experiment you can quickly collect up the cards and store them in the envelope that they are piled on. If you are doing more than 1 sort on the same day ensure you write which sort the cards belong to clearly on the envelopes.

The analysis

Once you’ve populated your spreadsheet with all your categories and cards from your sorts its time to start standardising your categories. When doing this you’re looking for 3 things, in my opinion:

1. The groupings of the cards into categories
Irrespective of the categories name, you want to look out for agreement by users that a set of cards should be categorised together.
2. The names given for the same category
Once you’ve identified that all your users put the cards bread, french stick and bagels into the same category then it’s time to review what they called those groups: 

  1. Breads
  2. Baked goods
  3. Bakery

You will have to come up with a naming convention and try to be consistent across all your categories. It may even be wise to consider search term traffic if you are looking to optimise your site for search, when picking a category name.

3. Spotting the dump category
When undertaking your analysis be mindful of the users that were in the groups. Consider their understanding of the content and if they are the target audience for all your content. I sometimes refer to something I have observed in the past as ‘the domain specialists vs. the dump category’.I’ll give you an example, where in a supermarket would you find Quorn (imitation meat mycoprotein) or a USB stick? If you’re a vegetarian or cook for vegetarians there is a good chance you know exactly where to go for the first item. Likewise, if you are into gadgets and are tech-savvy there is a good chance you’ll be able to find a USB stick in a supermarket and you’d know where to put it in a card sort. However, if you are neither a vegetarian or techie then you may simply give up. You may even create a category called ‘Misc’ and dump cards that you don’t know where they fit into it. 

More on analysis

There is quite a bit more to the analysis of card sorts, probably enough for another blog post on its own. As I suggested earlier, some of the best resources on ‘using the spreadsheet’ can be found on the Rosenfeld resource site.

Testing your new categories

So once you’re pretty happy that you have your categories, you have the items in them and you have determined a logical hierarchy, it’s time to test your IA. The following 2 methods can help to validate your IA:

Treejack

Treejack is an IA validation tool that allows you to test your IA without visual distractions.

You can create tasks for a user to undertake such as “You want to buy some fresh bread to make bacon sandwiches at the weekend, where would you locate the fresh bread?”.

The tool tracks the users path through the navigation and produces reports on their outcomes.

Task based testing

Once you have Treejack tested your IA it’s time to include it in any prototype that you would build during the design phase of the project. At Sigma we are big advocates or developing prototypes for testing, prior to sites or applications being developed.

Both these approaches will give you insight into how users interact with your categories and the content within them. Based on the results you may want to refine your IA further and re-test.

Conclusion

There are a lot of steps to this post and you may need to refer back to them at different times. The steps to a successful card sort are:

  1. Pick & prep your cards
  2. Run your experiment
  3. Analyse your results
  4. Test your findings

Card Sorting: Designing Usable Categories by Donna Spencer 

For more on Card Sorting I strongly recommend this book. In it, Donna describes how to plan and run a card sort, then analyse the results and apply the outcomes to your project.

It’s a great addition to the bookshelf and a great reference before, during and after a card sort.

[1] Spencer, Donna & Warfel, Todd. 2004. “Card sorting: a definitive guide.” [online]. Published on 7th April 2004. http://www.boxesandarrows.com